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Abstract 


The  Federal  Acquisition  Regulation  (FAR)  mandates  the  use  of  contractor  past 
performance  information  (PPI)  as  an  evaluation  factor  in  all  source  selections  involving 
negotiated  procurements  above  $1 ,000,000.  Different  agencies  within  Federal 
Government  have  lowered  the  dollar  threshold  to  as  little  as  $100,000  depending  on  the 
type  of  contract  action.  Using  PPI  as  a  factor  in  all  source  selections  was  a  tactic 
modeled  after  industry  best  practices.  The  Office  of  Federal  Procurement  Policy 
(OFPP)  envisioned  that  industry  PPI  collected  by  Federal  agencies  would  eventually  be 
automated,  maintained  on  secure  databases,  and  available  to  all  Government  agencies 
for  source-selection  purposes.  This  practice  would  eventually  lead  to  efficiencies  similar 
to  those  in  the  private  sector.  However,  along  the  way,  Government  and  private  sector 
industry  have  begun  to  disagree  about  how  PPI  is  collected  and  how  PPI  is  used. 
Industry  prefers  a  passive  system  of  collecting  delivery  and  quality  data  during  contract 
performance,  while  Federal  government  uses  both  a  passive  system  (similar  to  industry) 
as  well  as  an  active  system  of  pulling  PPI  during  contract  performance.  Industry  uses 
PPI  to  establish  and  maintain  a  preferred  vendor  list  from  which  to  solicit  bids,  quotes  or 
proposals,  while  government  uses  PPI  to  assess  risk  and  establish  vendor  responsibility 
in  a  full  and  open  competition  environment.  Contract  award  cycle-time  within  the 
Federal  Acquisition  process  is  more  than  double  that  of  the  private  sector  due  to  an 
evaluation  process  that  is  cumbersome,  time-consuming,  and  lacking  the  efficiencies 
enjoyed  by  private  industry.  Government  (the  DoD  in  particular)  has  recently  become 
more  curious  in  regards  to  industry  best  practices  and  how  those  practices  can  be 
implemented  in  the  government,  in  particular,  in  the  possibility  of  diminishing  the 
Government  Contracting  Officer’s  and  the  Source  Selection  team’s  added  burden.  This 
paper  will  explore  through  field  research  the  current  PPI  collection  and  evaluation 
process  used  by  the  DoD  and  by  those  employed  in  industry.  The  goal  behind  such 
research  is,  again,  industry  best  practices  and  improving  the  DoD’s  use  of  PPI  as  a  tool 
in  the  acquisition  processes. 
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CHAPTER  I:  INTRODUCTION 


A.  BACKGROUND 

The  Federal  government  is  in  a  continuous  process  to  reinvent  itself,  with  the  aim 
of  becoming  a  government  that  works  better  and  costs  less.  The  US  government  is  the 
largest  acquisition  organization  in  the  world,  with  expenditures  of  about  $200  billion  a 
year  for  commercial  goods  and  services.  This  is  one  third  of  the  Federal  discretionary 
budget  of  about  $600  billion.  How  well  the  Government’s  acquisition  teams  administer 
in-progress  contracts  and  discussions  with  vendors  about  their  current  performance 
determines,  to  a  great  extent,  how  well  agencies  can  achieve  their  mission  goals.  By 
increasing  attention  to  contractor  performance  on  in-progress  contracts  and  ensuring 
past  performance  data  is  readily  available  for  source  selection  teams,  agencies  and 
subsequent  contracting  activities  are  reaping  two  benefits:  1 )  better  current  performance 
because  of  the  active  dialog  between  the  contractor  and  the  government;  and  2)  better 
ability  to  select  high-quality  contractors  [vendors]  because  vendors  know  the 
assessments  will  be  used  in  future  contract  award  decisions.1  One  of  the  key  evaluation 
criteria  used  by  the  Federal  Government  in  the  source  selection  process  is  the 
measurement  of  a  potential  vendor’s  past  performance  history,  or  put  differently,  the  risk 
of  contractor  non-performance.  OFPP  Policy  Letter  92-5  “Past  Performance 
Information”  began  this  process  in  1992  by  requiring  that  past  performance  be  included 
as  a  major  factor  in  Federal  source  selections.  The  requirement  applies  to 
procurements  with  an  anticipated  total  contract  value  greater  than  $1  million  and  relies 
upon  the  information  collected  during  contract  performance  and  an  end-of-contract 
report  (report  card).  The  collection  and  use  of  past  performance  information  is  viewed 
by  both  government  and  industry  as  a  burdensome  process;  however,  past 
performance  is  a  key  indicator  of  risk.  Failure  to  consider  a  potential  vendor’s  past 
performance  can  significantly  increase  the  risk  of  contract  delays  or  non-compliance. 


1  Office  of  Federal  Procurement  Policy,  Office  of  Management  and  Budget,  Executive  Office  of  the 
President,  Best  Practices  for  Collecting  and  Using  Current  and  Past  Performance  Information,  May  2000, 
4. 
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B.  SCOPE  OF  STUDY 


This  research  paper  will  examine  the  collection  and  use  of  Past  Performance 
Information  (PPI)  from  three  segments:  1)  Government,  and  in  particular  Department  of 
Defense  (DoD),  2)  Defense-focused  industry,  and  3)  Non-Defense  focused  industry. 

The  scope  of  this  study  is  strictly  a  field  research  effort.  It  will  be  limited  to  data 
collected  during  visits  to  each  of  the  eight  sites  with  emphasis  on  what  is  actually 
happening  in  the  acquisition  environment;  likewise,  it  will  include  recommendations  for 
improvement.  Data  was  collected  using  the  survey/questionnaire  shown  in  Appendix  1 . 
Dollar  thresholds  for  public-sector  past-performance  collection  are  shown  in  Appendix  2, 
and  results  from  each  of  the  interviews  are  summarized  in  Appendix  3-10.  The 
interview  data  will  be  the  basis  for  discussion,  analysis  and  recommendations. 

C.  DEFINITIONS 

The  following  terms  are  used  frequently  throughout  this  paper.  It  is,  therefore, 
prudent  to  gain  an  understanding  of  these  terms  in  order  to  follow  the  text. 

Best  Value:  Tradeoff  process.  FAR  Part  15.1 01 -1(c)  This  process  permits 
tradeoffs  among  cost  or  price  and  non-cost  factors  and  allows  the  Government  to 
accept  other  than  the  lowest  priced  proposal.  The  perceived  benefits  of  the  higher 
priced  proposal  shall  merit  the  additional  cost,  and  the  rationale  for  tradeoffs  must  be 
documented  in  the  file  in  accordance  with  15.406. 

PCO:  Procuring  Contracting  Officer.  A  government,  procuring  official  warranted 
to  award  government  contracts  and  negotiate  terms  and  conditions  with  private  sector 
firms. 


Private  Sector:  For  purposes  of  this  research  effort,  Private  Sector  refers  to 
firms  who  conduct  business  for  profit. 

Public  Sector:  For  purposes  of  this  research  effort,  Public  Sector  refers  to  all 
Federal,  State,  and  local  government  agencies  and  activities. 
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Quote,  bid,  or  proposal:  Refers  to  an  offer  from  a  private  sector  vendor  in 
response  to  a  solicitation  to  provide  supplies,  services,  or  construction. 

Solicitation:  any  request  to  submit  offers  or  quotations  to  the  government.  Also 
referred  to  as  a  Request  for  Proposal  (RFP). 

Vendor:  Used  interchangeably  with  contractor  or  industry  firm  from  the  private 
sector  that  provides  bids,  quotes  or  proposals  for  the  purpose  of  receiving  government- 
awarded  contracts. 

D.  RESEARCH  QUESTIONS 

In  order  to  focus  on  a  specific  effort,  this  study  will  pursue  two  basic  research 
questions,  as  follows: 

1.  What  are  the  principal  issues  involved  in  using  PPI  in  the  source-selection  process? 

2.  How  might  an  assessment  of  industry  models  of  past-performance  evaluation  assist 
in  improving  the  DoD’s  use  of  past  performance  as  an  evaluation  criterion? 
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CHAPTER  II.  LITERATURE  REVIEW 


A.  REGULATORY  &  POLICY  GUIDANCE,  INCLUDING  DoD  AND 
OTHER  AGENCIES 

Within  the  Federal  Government,  there  is  a  body  of  mandatory  and  discretionary 
guidance  published  by  the  Office  of  Federal  Procurement  Policy,  the  Office  of  the 
Secretary  of  Defense,  and  by  each  of  the  Services.  References  reviewed  and  used 
heavily  in  this  paper  include: 

•  Federal  Acquisition  Regulation  (FAR) 

•  Defense  Federal  Acquisition  Regulation  Supplement  (DFARS) 

•  Best  Practices  for  Collecting  and  Using  Current  and  Past  Performance  Information, 
Office  of  Federal  Procurement  Policy  (OFPP),  May  2000 

•  A  guide  to  Collection  and  Use  of  Past  Performance  Information  (Version  3),  Office  of 
the  Under  Secretary  of  Defense  for  Acquisition,  Technology  &  Logistics  (ASN:ATL), 
May  2003 

•  Uniform  Commercial  Code  (UCC) 

The  above  references  provided  the  procedures  which  are  utilized  across  Federal 
Government  to  collect,  analyze,  store,  and  retrieve  contractor  past  performance  for 
purposes  of  source  selections.  All  references  listed  above  (with  the  exception  of  the 
UCC)  reference  the  FAR,  and  essentially  say  the  same  thing  as  that  document  does. 
The  UCC,  on  the  other  hand,  guides  private  industry,  yet  only  addresses  the  sale  of 
goods. 

1.  Statutory  and  Regulatory  Guidance 

The  1994  Federal  Acquisition  Streamlining  Acquisition  Act  (FASA)  signaled  a  “sea 
change”  in  Federal  acquisition;  FASA  was  signed  into  law  by  President  Clinton  on  13 
October  1994  (P.L.  103-355).  In  FASA,  Congress  acknowledged  it  is  appropriate  and 
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relevant  for  the  government  to  consider  a  [vendor’s]  past  performance  in  evaluating 
whether  that  vendor  should  receive  future  work. 2 

2.  FAR  Part  6.101  Policy. 

(a)  10  U.S.C.  2304  and  41  U.S.C.  253  require,  with  certain  limited 
exceptions  (see  Subparts  6.2  and  6.3),  that  contracting  officers  shall 
promote  and  provide  for  full  and  open  competition  in  soliciting  offers 
and  awarding  Government  contracts. 

(b)  Contracting  officers  shall  provide  for  full  and  open  competition 
through  use  of  the  competitive  procedure(s)  contained  in  this  subpart 
that  are  best  suited  to  the  circumstances  of  the  contract  action  and 
consistent  with  the  need  to  fulfill  the  Government's  requirements 
efficiently  (10  U.S.C.  2304  and  41  U.S.C.  253). 

3.  FAR  Part  1 .102  Statement  of  guiding  principles  for  the  Federal  Acquisition 
System. 

(b)  The  Federal  Acquisition  System  will  satisfy  the  needs  of  its 
customers  in  terms  of  cost,  quality  and  timeliness  by  ...  using 
contractors  who  have  a  track  record  of  successful  past  performance  or 
who  demonstrate  a  current  superior  ability  to  perform. 


2  Ibid.,  4. 
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CHAPTER  III.  RESEARCH  METHODOLOGY 


A.  DESCRIPTION  OF  THE  STUDY 

The  DoD  has  structured  an  elaborate  system  for  the  collection  and  use  of 
contractor  past  performance  information  (PPI)  in  the  source-selection  phase  of  an 
acquisition  process.  The  DoD  system  was  modeled  after  best  commercial  practices 
and  was  to  employ  sound  business  judgment.  A  comparison,  however,  of  the  methods 
used  by  the  DoD  with  those  used  by  industry  now  show  striking  differences.  This  study 
will  compare  the  DoD’s  and  industry’s  past  performance  measures  with  the  aim  of 
improving  the  DoD  system  to  better  reflect  sound  business  practices.  Various  firms 
within  industry  will  be  contacted  to  determine  the  difficulties  they  have  experienced  with 
their  past  performance  system  and  its  use  in  their  source-selection  process. 

The  U.  S.  Navy  has  adopted,  to  a  large  extent,  the  U.  S.  Air  Force’s  program 
known  as  the  Contractor  Performance  Assessment  Review  System  (CPARS)  for  PPI 
data  collection  and  application  in  the  source-selection  process.  Three  of  the  four 
government  activities  interviewed  during  this  research  effort  were  U.  S.  Navy  activities. 
The  fourth  activity  was  the  U.  S.  Army,  which  also  uses  CPARS.  CPARS  will  be 
compared  to  industry  models  with  a  goal  of  improving  the  Navy’s  application  of  past 
performance  data  and  improving  cycle  time  for  contract  awards. 

B.  RESEARCH  DESIGN 

This  study  was  a  field  research  effort.  The  investigator  conducted  on-site 
interviews/surveys  at  various  DoD  and  industry  sites.  While  at  each  site,  the 
investigator  discussed  aspects  of  each  respective  acquisition  process  as  it  pertains  to 
the  collection  and  evaluation  of  a  vendor's  past  performance  information.  The  on-site 
interviews/surveys  were  conducted  with  key  members  of  the  acquisition  process  within 
each  activity  or  firm.  These  members  included  but  were  not  limited  to  one  or  more  of 
the  following: 
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•  Procuring  Contracting  Officer  (PCO) 

•  Chiefs  or  Deputy  Chiefs  of  the  Contracting  Office  (CCO) 

•  Source  Selection  Authority  (SSA) 

•  Source  Selection  Evaluation  Board  (SSEB)  team  members 

•  Source  Selection  Advisory  Counsel  (SSAC)  team  members 

•  Directors  of  Business  Development 

•  Directors  of  Purchasing 

•  Directors  of  Legal/Contracts 

Although  not  statistically  significant,  field  research  does  become  a  data  point  as 
to  what  practices  are  currently  in-force  and  how  they  are  working.  Table  1  provides  the 
research  design  used  in  this  study. 


Activity 

Interview/ 

Analysis  & 

Survey  results 

Comparisons 

Public  Sector: 

1)  Naval  Sea  Systems  Command 

♦ 

♦ 

2)  Naval  Air  Systems  Command 

♦ 

♦ 

3)  Fleet  &  Industrial  Supply  Center,  Norfolk 

♦ 

♦ 

4)  Tank  Automotive  Command 

♦ 

♦ 

Private  Sector  (Defense  focused) 

1)  Industry  A  -  Lockheed  Martin,  Space  Systems  Company 

♦ 

♦ 

2)  Industry  B  -  Northrop  Grumman  Newport  News 

♦ 

♦ 

Private  Sector  (non-Defense  focused) 

3)  Industry  A  -  Opal  Soft  Inc 

♦ 

♦ 

4)  Industry  B  -  Maersk  Line  Limited 

♦ 

♦ 

Table  1:  Past  Performance  Information — Research  Model 


As  stated  previously,  the  field  research  for  this  study  was  conducted  using  a 
prepared  Interview/Survey  questionnaire.  A  copy  of  the  Interview/Survey  questionnaire 
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is  located  in  Appendix  1 .  Appendix  2  contains  the  dollar  thresholds  for  the  use  of  PPI. 
When  conducting  field  interviews,  the  underlying  acquisition  scenario  was  assumed: 

•  Eventual  contract  value  would  exceed  $1 M  ($1 00,000  for  smaller  firms); 

•  Simplified  Acquisition  procedures  would  not  be  employed; 

•  The  acquisition  would  use  a  Best  Value-Tradeoff  type  approach  (see  definitions); 

•  The  major  factors  outlined  in  the  solicitation  would  include  Technical/Quality,  Past 
Performance,  and  Cost/Price. 
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CHAPTER  IV.  RESULTS  AND  ANALYSIS  OF  FIELD 
RESEARCH 


Using  the  research  model  outlined  in  Chapter  III,  the  investigator  systematically 
interviewed/surveyed  key  acquisition  team  members  in  each  of  the  three  sectors,  as 
follows:  1 )  Public  Sector  (DoD),  2)  Private  Sector  (Firms  with  roughly  greater  than  50% 
defense-based  contracts),  and  3)  Private  Sector  (Firms  with  roughly  less  than  50% 
defense-based  contracts).  All  DoD  activities  and  Industry  firms  were 
interviewed/surveyed  using  the  same  questionnaire.  Using  one  questionnaire  was 
useful  in  drawing  comparisons  and  a  baseline.  Tables  2  through  12  summarize  this 
effort.  Following  is  the  narrative  results  of  this  field  research  effort  (see  Appendix  3-10). 

A.  PUBLIC  SECTOR:  WHAT  IS  THE  DoD  DOING? 

Four  public  sector  DoD  acquisition  activities  were  interviewed/surveyed — three 
supporting  the  U.  S.  Navy,  and  one  supporting  the  U.  S.  Army.  The  purpose  of  using 
four  public  sector  DoD  activities  was  to  gain  a  benchmark,  or  consensus,  by  which  to 
gauge  differences  in  the  way  public  and  private  sector  perform  the  acquisition  process, 
and  more  importantly,  how  each  collects  and  evaluates  past  performance  information 
(PPI)  as  a  factor  in  elimination  of  an  offer  from  the  competitive  range  prior  to  contract 
award,  or  as  a  factor  in  non-selection  for  contract  award.  The  following  is  a  short 
description  of  each  activity  interviewed. 
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1.  Public  Sector  (DoD)  Activity  #1:  Naval  Sea  System  Command  (NAVSEA) 

Located  at  the  Navy  Yard  in  Washington,  D.C.,  NAVSEA  is  responsible  for  the 
procurement,  maintenance,  overhaul,  conversion  &  repair  of  all  ships  and 
submarines  in  the  U.  S.  Navy  inventory.  Interview/survey  data  is  found  in 
Appendix  3.  New  programs  include,  but  are  not  limited  to  the  following3: 


FY2003 

FY2004 

FY2005 

CVN-77 

Aircraft  Carrier 

849.3 

1516.1 

978.9 

DD(X) 

DD(X)  Destroyer 

916.3 

1088.9 

1450.6 

DDG-51 

AEGIS  Destroyer 

3012.4 

3406.5 

3591.5 

LCS 

Littoral  Combat  Ship 

35.3 

166.2 

352.1 

LDP-17 

Amphib  Transp  Ship  Dock 

594.0 

1325.5 

975.6 

NSSN 

Virginia  Class  Submarine 

2335.4 

2514.3 

2596.3 

RCOH 

CVN  Refueling  Compl  Ovhl 

217.3 

221.0 

333.1 

SSGN 

SSBN  to  SSGN  Conversion 

1183.3 

1227.5 

658.4 

T-AKE 

Aux  Dry  Cargo  Ship 

386.0 

722.3 

768.4 

Table  2.  Public  Sector  (DoD)  Activity  #1  Naval  Sea  System  Command  (NAVSEA) 

(Dollars  in  Millions) 


2.  Public  Sector  (DoD)  Activity  #2:  Naval  Air  Systems  Command  (NAVAIR) 

Located  in  Patuxet  River,  MD,  NAVAIR  supports  the  planning,  acquisition, 
production,  operational  support,  overhaul,  upgrade,  repair  of  all  aircraft  and  aircraft 
weapon  systems  for  the  U.S.  Navy.  Interview/survey  data  is  found  in  Appendix  4. 
Major  acquisitions  include4: 


3  Defense  AT&L,  A  publication  of  the  Defense  Acquisition  University:  In  the  news ,  May-June  2004:  68-69 

4  Ibid.,  68-69. 
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FY2003 

FY2004 

FY2005 

E-2C 

Hawkeye 

393.6 

570.1 

845.0 

EA-6B 

Prowler 

368.1 

370.2 

199.7 

FA-18E/F 

Hornet 

3401.1 

3217.8 

3120.4 

H-1 

USMC  H-1  Upgrades 

232.2 

399.5 

332.2 

MH-60R 

Helicopter 

207.1 

461.7 

487.9 

MH-60S 

Helicopter 

375.7 

467.0 

482.0 

T-45TS 

Goshawk 

218.2 

336.7 

253.6 

Table  3.  Public  Sector  (DoD)  Activity  #2:  Naval  Air  Systems  Command  (NAVAIR) 


(Dollars  in  Millions) 


3.  Public  Sector  (DoD)  Activity  #3:  Fleet  and  Industrial  Supply  Center,  Norfolk 
Naval  Shipyard  Annex  (FISC/NNSY  Annex) 

Located  in  Portsmouth,  VA,  FISC/NNSY  Annex  is  a  field  contracting  activity  for  the 
U.  S.  Navy  and  was  selected  for  this  study  due  to  the  unique  nature  of  the  activity  it 
supports,  and  the  use  of  a  passive  contractor  past  performance  collection  and 
evaluation  system  which  mimics  that  of  the  private  sector.  Interview/survey  data  is 
summarized  in  Appendix  5.  Major  projects  include: 


CVN 

Aircraft  Carrier 

Overhaul,  Conversion  and  Repair 

LHA/LHD 

Amphib  Assault  Ships 

Overhaul,  Conversion  and  Repair 

SSN 

Attack  Submarine 

Overhaul,  Conversion  and  Repair 

SSBN 

FBM  Submarine 

Overhaul,  Re-fuel,  Conversion  to  SSGN 

Table  4.  Public  Sector  (DoD)  Activity  #3:  Fleet  and  Industrial  Supply  Center,  Norfolk  Naval 

Shipyard  Annex  (FISC/NNSY  Annex). 
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4.  Public  Sector  (DoD)  Activity  #3:  Tank  Automotive  Command  (TACOM) 

Located  in  Warren,  Ml,  TACOM  is  responsible  for  the  planning,  acquisition,  fielding, 
and  support  for  the  majority  of  the  land  combat  vehicles  for  the  U.  S.  Army. 
Interview/survey  data  is  found  in  Appendix  6.  Major  acquisitions  include5: 


FY2003 

FY2004 

FY2005 

FCS 

Future  Combat  System 

370.0 

1638.6 

3198.1 

M1A2 

Abrams  Tank  Upgrade 

551.1 

207.9 

308.3 

M2  A3 

Bradley  Base  Sustainment 

437.4 

344.5 

71.4 

IAV 

Interim  Armored  Vehicle  (Stryker) 

930.3 

1043.4 

957.0 

Table  5.  Public  Sector  (DoD)  Activity  #3:  Tank  Automotive  Command  (TACOM) 


(Dollars  in  Millions) 


The  above-listed  DoD  acquisition  activities  use  CPARS  as  their  primary  past  - 
performance  collection  tool.  Contracts  are  awarded  in  accordance  with  the  Federal 
Acquisition  Regulation  (FAR),  Defense  Federal  Acquisition  Regulation  Supplement 
(DFARS),  applicable  policy  guidance,  and  internal  acquisition  procedures.  CPARS  will 
be  discussed  later  in  this  chapter. 

In  an  effort  to  automate  and  share  past  performance  information,  the  DoD  has 
made  available  the  ability  to  access  PPI  in  a  common  database  structure.6  This 
database  will  be  described  later  in  this  chapter.  DoD  acquisition  activities  must  collect 
PPI  on  1)  in  progress  contract  actions  and  2)  contract  actions  which  have  been 
completed.  An  analysis  of  differences  between  the  DoD  and  its  industry  counterparts 
will  be  conducted  in  this  chapter. 


5  Ibid.,  68-69. 

6  Office  of  the  Under  Secretary  of  Defense,  Acquisition,  Technology  and  Logistics,  Policy  Letter 
DPAP/EB,  May  3,  2004,  5. 
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B.  INDUSTRY  (DEFENSE-FOCUSED  FIRMS):  WHAT  IS  THE 
DEFENSE  INDUSTRY  DOING? 


Two  private-sector  activities  were  interviewed/surveyed.  Both  firms  support  the 
DoD  as  their  primary  customer.  Surprisingly,  both  activities  look  and  operate  like  their 
counterparts  in  the  public  sector  in  regards  to  acquisitions  of  supplies  or  services.  The 
acquisition  cycles  of  both  firms  follow  similar  paths,  from  acquisition  planning  through 
contract  award.  So,  too,  does  the  collection  and  use  of  past  performance  information. 
Both  firms  use  a  scorecard  approach,  which  bares  stark  similarities  to  the  U.  S.  Navy’s 
Red,  Yellow,  and  Green  (RYG)  program  for  evaluating  supplier  performance  based  on 
quality  and  delivery.  Both  RYG  and  Scorecard  will  be  compared  and  analyzed  in  this 
chapter. 

Private  Sector  (Defense-focused)  Activity  #1 :  Lockheed  Martin  Space  Systems 
Company  (LM) 

The  division  of  LM  interviewed  for  this  research  is  located  in  Sunnyvale,  CA  at  the 
Moffet  Field  complex.  LM  Space  Systems  Company  designs,  produces  and  integrates 
launch  vehicles  and  systems,  spacecraft  for  telecommunications,  remote  sensing  and 
space  science,  as  well  as  missile  systems  for  defensive  and  strategic  missions. 

Space  Systems  comprises  three  joint  ventures:  United  Space  Alliance  manages 
and  conducts  space  operations  work  involving  the  operation  and  maintenance  of  multi¬ 
purpose  space  systems,  and  is  the  prime  contractor  for  NASA's  Space  Shuttle  program; 
Space  Imaging  supplies  earth  information  derived  from  space  imagery  and  aerial 
photography  to  commercial  and  government  customers;  and  International  Launch 
Services  is  the  leader  in  the  launch  services  arena  with  a  solid  history  of  more  than  80 
years  of  combined  experience  and  800+  combined  flights  on  the  two  premier  launch 
vehicles — Atlas  and  Proton.  LM  Space  Systems  Company  is  a  leading  provider  of  the 
world's  most  advanced  and  secure  defense  communications,  navigation,  and 
meteorological  systems  serving  the  U.S.  and  allies  around  the  globe.  Military  Space 
includes  products  like  Milstar,  Advanced  EHF  military  communications  satellite  system, 
Space  Based  Infrared  Systems  (SBIRS)  and  Global  Positioning  Systems  (GPS). 
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Since  1955,  Space  Systems  Company  has  been  the  Navy's  prime  strategic 
missile  contractor  and  missile  system  manager.  The  six  generations  of  Fleet  Ballistic 
Missiles  include  Polaris,  Poseidon  and  the  current  Trident  II  D5.7  Interview/survey  data 
is  summarized  in  Appendix  7. 


Picture  1. Trident  D-2  SLBM 


Private  Sector  (Defense-focused)  Activity  #2:  Northrop  Grumman  Newport  News 
(NGNN) 

Located  in  Newport  News,  VA,  NGNN  designs,  builds,  overhauls,  and  repairs  a  wide 
variety  of  ships  for  the  U.S.  Navy  and  commercial  customers.  Today,  Newport  News  is 
the  nation's  sole  designer,  builder,  and  refueler  of  nuclear-powered  aircraft  carriers; 
also,  it  is  one  of  only  two  companies  capable  of  designing  and  building  nuclear-powered 
submarines.  The  company  also  provides  after-market  services  for  a  wide  array  of  naval 
and  commercial  vessels,  and  in  November  2001 ,  became  a  sector  of  Northrop 
Grumman  Corporation.  Summary  follows: 

•  Sole  supplier  of  U.S.  Navy  aircraft  carriers,  the  world’s  largest  warships. 

•  Builder  of  25  of  the  Navy  attack  submarines  in  use  today — and  1 1  of  the  nation’s  1 2 
active  aircraft  carriers. 

•  Home  of  the  Western  Hemisphere’s  largest  dry  dock  and  crane. 

•  Exclusive  provider  of  refueling  services  for  nuclear-powered  aircraft  carriers. 

•  Largest  industrial  employer  in  Virginia. 


7  Lockheed  Martin  Space  Systems  Company,  official  website,  www.lockheedmartin.com  visited  Dec  2004 
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Largest  non-governmental  provider  of  fleet  maintenance  services  to  the  Navy. 8 


Interview/survey  data  is  summarized  in  Appendix  8. 


Picture  2.  Nimitz-class  aircraft  carrier 

C.  INDUSTRY  (NON-DEFENSE-FOCUSED  FIRMS):  WHAT  IS 
PRIVATE  INDUSTRY  DOING? 

Two  additional  private  sector  activities  were  interviewed/surveyed.  Both  support 
the  DoD  and  private  industry  customers.  Both  firms  exhibit  a  high  correlation  to  the 
public  sector  in  their  acquisition  approach,  yet  illustrate  less  correlation  in  their  approach 
to  collection  and  use  of  contractor  past-performance  information.  An  analysis  of  these 
similarities  and  differences  will  be  conducted  later  in  this  chapter. 

Private  Sector  (Non-Defense-Focused)  Activity  #3:  Opalsoft 

Located  in  San  Jose,  CA,  Opalsoft  is  a  small  business  (8a  firm).  OpalSoft  is  a 
consultancy  and  integration  firm  providing  specialized  IT  services  including  Software 
Engineering  and  Development,  System  Integration  and  Technology  Consulting. 

Services  in  this  area  are  focused  on: 

•  Product  Engineering 


8  Northrop  Grumman  Newport  News  (NGNN),  official  website:  www.nn.northropqrumman.com  visited  Dec 
2004 


NPS 


PRAfcSTANTlA  rER  i 


ACQUISITION  RESEARCH 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY 
NAVAL  POSTGRADUATE  SCHOOL 


-  15- 


•  Application  Development 

•  Re-Engineering  Legacy  Applications 

•  Application  Support 

Projects  range  from  complex  e-business  and  web  development  initiatives  to 
enterprise-wide  application  development  to  new  product  introduction.  OpalSoft  has 
capabilities  for  engaging  customers  at  every  stage  of  a  software-development  life-cycle: 
Conceptualization,  Development,  Implementation  and  Support.  OpalSoft  provides  a 
flexible  model  for  execution  of  the  “just-for-you”  solution  tailored  to  meet  specific  needs 
of  clients.  The  Delivery  Model  provides  for  execution  of  projects,  either  onsite  (at  client 
site)  or  offsite  (OpalSoft  facilities  in  San  Jose)  or  a  combination  of  both.  It  ensures 
complete  transparency  of  the  development  effort  following  predefined  objectives, 
milestones,  QA  &  Testing,  acceptance  criteria  and  delivery.  It  is  driven  by  critical  factors 
such  as  time-to-market,  technologies  involved,  integration  issues,  requirement  of 
interactive  and  iterative  process,  requirement  of  client  interface  etc.9  Interview/survey 
data  is  summarized  in  Appendix  9. 

Private  Sector  (Non-Defense-Focused)  Activity  #4:  Maersk  Line,  Ltd. 

Headquartered  in  Norfolk,  Virginia,  Maersk  Line,  Ltd.  (MLL),  provides  maritime  services 
and  logistics  capabilities.  Although  MLL  is  categorized  as  non-defense  focused,  their 
government  customer  base  is  solid  and  growing.  MLL  specializes  in  ship  owning, 
vessel  life-cycle  management  services,  and  integrated  defense  logistics  solutions. 
Maersk  Line,  Ltd.,  has  a  diverse  fleet  of  vessels,  all  of  which  are  registered  in  the  United 
States,  and  crewed  by  U.S.  Merchant  Mariners.  Vessels  are  engaged  globally,  providing 
sealift  support  for  the  U.S.  Military,  ocean  surveillance,  charter  and  inducement 
services,  liner  services,  and  access  to  the  world's  most  comprehensive  transportation 
network  through  Maersk  Sealand. 


9  Opalsoft  official  website,  www.opalsoft.com  visited  Dec  2004 
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Commercial 


Maersk  Line,  Limited  owns  or  operates  a  total  of  twenty-six  US  Flag  vessels 
engaged  in  commercial  service.  Twenty-three  container  vessels  are  deployed  in  liner 
service  with  the  parent  company,  Maersk  Sealand,  which  provides  unparalleled  access 
to  US  Flag  service  and  the  world's  most  comprehensive  transportation  network.  Three 
additional  US  Flag  vessels  operate  in  worldwide  service,  and  can  accommodate  diverse 
cargoes  with  break-bulk,  container  and  roll-on,  roll-off  capability.  These  vessels  are 
currently  deployed  in  the  following  trades: 

•  Maersk  Arizona  (or  substitute):  US  East  Coast  and  US  Gulf  to  the  Middle  East  and 
West  Central  Asia 

•  Maersk  Alaska  (or  substitute):  US  to  the  Far  East 

•  Maersk  Constellation  (or  substitute):  US  East  Coast  and  US  Gulf  to  East  and  West 
Africa 


Picture  3.  Maersk  Line  Ltd.,  Container  ship 


Military 

Maersk  Line,  Ltd.,  owns  and  operates  US  Flag  vessels  under  exclusive  contact 
with  the  Military  Sealift  Command  in  support  of  the  US  Military's  sealift  requirements. 
These  vessels  operate  in  a  pre-positioned  capacity;  they  provide  transport  and  storage 
for  rolling  stock,  break-bulk,  containers  and  ammunition  for  the  U.S.  Army  and  Marine 
Corps,  as  well  as  ocean  surveillance  and  radar  support  to  the  US  Navy.10 
Interview/survey  data  is  summarized  in  Appendix  10. 


10  Maersk  Line,  Limited  official  website,  www.maersklinelimited.com  visited  Dec  2004 
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D.  OBSERVATIONS  AND  ANALYSIS  OF  FIELD  RESEARCH 


For  the  purpose  of  direct  comparisons  between  each  activity/site  interviewed, 
Tables  6-12  have  been  prepared.  Each  table  correlates  to  a  group  of  interview 
questions,  as  follows: 

•  Section  1 — General 

•  Section  2 — Collection  of  Past  Performance  Information 

•  Section  3 — Evaluation  of  Past  Performance  Information 


1.  Observations  and  Analysis:  General 

DoD-Related  Mission,  Interview  Question  #1.  As  indicated  in  Table  6  (pg.  20), 
all  activities  interviewed  are  defense-focused  with  the  exception  of  Industry  firms  three 
(Opalsoft)  and  four  (Maersk  Line,  Ltd.).  Industry  firms  three  and  four  both  compete  for 
government  contracts,  yet  both  have  a  significant  private-sector  customer  base.  This 
observation  will  have  a  significant  flow-down  affect  as  additional  observations  are 
discussed  and  analyzed. 

Contracts  Branch/Directorate,  Interview  Question  #2.  Largely,  the  activities 
interviewed  place  the  procurement  departments  in  the  first  tier  of  their  respective 
organizations,  preferring  to  “farm  out”  their  acquisition  workforce  to  different  programs, 
projects  or  individual  acquisitions.  Of  the  remaining  activities,  both  place  their 
contracting/purchasing  divisions  under  the  larger  Supply  Department  (this  is  also  the 
case  with  FISC,  NNSY),  or  they  place  their  contracting/purchasing  divisions  under  the 
Senior  Vice  President  for  Operations  as  is  the  case  with  NGNN.  Both  the  NNSY  Supply 
Officer  and  the  Senior  VP-Operations  are  in  the  first  tier  of  their  activities.  The  data 
suggests  that  both  public  sector  and  private  sectors  place  a  high  value  on  the 
procurement  and  acquisition  functions  within  their  organizations,  suggesting  the 
procurement  function  is  an  integral  part  of  business  operations.  The  data  also  suggests 
that  both  public  and  private  sectors  prefer  “centralized  management”  of  procurements  to 
stand  alone  operations,  which  support  individual  projects  or  teams.  Although  this  may 
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sound  contradictory,  procurement  departments  within  both  the  public  and  the  private 
sector  who  “farm  out”  their  acquisition  workforce  to  different  projects  continue  to  retain 
administrative  and  procedural  control  at  all  times.  For  example,  a  high  value  acquisition 
would  still  have  to  pass  a  “centralized”  contracts  review  board  (CRB)  prior  to  issuing  the 
solicitation  and/or  awarding  the  contract. 

The  researcher  offers  one  explanation  for  the  tendency  to  place  the  acquisition 
and  procurement  function  in  the  first  or  second  tier  of  an  organization,  with  significant 
management  oversight.  The  tendency  towards  centralization  is  tied  to  the  potential  for 
vendor  protests,  senate  and  congressional  inquiries,  and  litigation  in  both  public  and 
private  sectors.  General  Accounting  Office  (GAO)  protests  and  congressional  inquiries 
normally  occur  prior  to  contract  award  or  upon  news  that  a  contract  has  been  awarded. 
Protests  and  inquiries  are  time  consuming,  extremely  disruptive,  and  almost  always 
result  in  schedule  delays.  Litigation,  on  the  other  hand,  normally  occurs  in  the  post¬ 
award  between  the  vendor  and  the  procuring  activity.  In  either  case,  neither  party  is 
typically  staffed  to  handle  protests  and  large-scale  litigation  without  significant  disruption 
to  daily  operations.  Although  the  public  sector  prevails  in  roughly  two  thirds  of  the 
cases  decided  in  a  formal  litigation  process11  both  parties  spend  extraordinary  man¬ 
hours  in  preparation  at  both  the  working  level  and  at  the  management  level  in 
preparation  for  addressing  claims  and  protests.  For  this  reason,  management,  which 
many  times  include  on-staff  legal  counsel,  inject  themselves  into  the  procurement  and 
acquisition  process  to  head-off  potential  protests  and  litigation.  It  is  the  activities  or 
firms  who  decentralize  important  procurement  decisions  and  disengage  legal  counsel 
until  late  in  the  process  that  typically  receive  unfavorable  decisions  in  the  court 
systems.12 


11  Roger  D.  Lord,  Analysis  of  Contract  Disputes  Resolved  by  the  Armed  Services  Board  of  Contract 
Appeals  (ASBCA)  between  January  1998  and  June  1999:  Master’s  Thesis  (Monterey,  CA:  Naval 
Postgraduate  School,  December  1999),  64. 

12  Ibid.,  102. 
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Research 

Question 

Govt 

Acty  1 

NAVAIR 

Govt 

Acty  2 

NAVSEA 

Govt 

Acty  3 

FISC  / 

NNSY 

Govt 

Acty  4 

TACOM 

Industry 

Firm  1 

LOCK¬ 

HEED 

Industry 

Firm  2 

NORTH¬ 

ROP 

Industry 

Firm  3 

OPAL 

SOFT 

Industry 

Firm  4 

MAERSK 

LINE 

Section  1 
General 

1 .  Mission  is 
primarily  DoD 
Related? 

♦ 

♦ 

♦ 

♦ 

♦ 

♦ 

2.  Does  the 
Contracts 

Branch/ 

Directorate  fit  in 
first  tier  of  your 
organization? 

♦ 

♦ 

♦ 

♦ 

♦ 

♦ 

Table  6.  Summary  of  Findings — General 


2.  Observations  and  Analysis:  Collection  of  Past  Performance 
Information 

Types  of  PPI  Data  Collected,  Interview  Question  #3.  Surprisingly,  Industry’s 
collection  of  various  elements  of  Past  Performance  Information  (PPI)  mirrors  that  of  the 
Public  Sector  as  shown  in  Table  7.  Quality  and  delivery  are  the  common  metrics 
tracked  by  both  public  sectors  (Government)  and  private  sector  (Industry).  Three  of  the 
four  private-sector  firms  interviewed  during  this  study  used  a  Scorecard  method  of 
tracking  PPI.  Typically  the  scorecard  method  is  broken  down  into  two  factors:  1 )  quality 
and  2)  delivery.  For  higher  dollar  value  procurements  in  both  government  and  industry, 
cost  control  and  business  relations  are  also  tracked.  Within  Government  and  Industry, 
all  vendors  evaluated  under  a  scorecard  system  are  given  periodic  feedback  as  to  their 
performance  during  the  period  (e.g.  quarter,  month,  continuous). 


Industry  favors  a  centralized  supply-receiving  area  that  is  co-located  with  a 
branch  of  quality  control  personnel,  thereby  capturing  both  late  deliveries  and  shipping 
damages,  or  other  obvious  quality  defects.  The  only  system  similar  to  Industries’ 
Scorecard  system  is  the  U.  S.  Navy’s  Red,  Yellow,  Green  (RYG)  System.  RYG  and 
the  differences  between  active  and  passive  PPI  collection  systems  will  be  explained 
later  in  this  chapter.  Table  7  (next  page)  summarizes  the  similarities  and  differences  in 
government  and  industry  PPI  collection. 
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<$100K  Commodities, 

Small  Contracts 

>$100K  Capitol  Goods  & 
Services,  Large  Contracts 

Government 

Navy  Red,  Yellow,  Green 
Program 

■  Quality 

■  Delivery 

(Passive  PPI  collection) 

CPARS  Program 

■  Business  Relations 

■  Quality 

■  Delivery 

■  Cost  Control 

(Active  PPI  Collection) 

Industry 

Scorecard 

■  Quality 

■  Delivery 

(Passive  PPI  collection) 

Scorecard 

■  Quality 

■  Delivery 

Manual  Files 

■  Business  Relations 

■  Cost  Control 

(Passive  PPI  collection) 

Table  7.  Collection  of  Past  Performance  Information  Metrics 


Why  are  quality  and  delivery  important  metrics  to  track?  Within  government,  and 
particularly  the  DoD,  the  development  of  new  weapons  systems  and  high  combat 
readiness  of  existing  weapon  systems  is  the  terminal  objective.  In  industry,  the  terminal 
objective  is  increased  revenue,  higher  profit  and  a  reputation  based  on  a  high-quality 
product.  Quality  and  delivery  of  required  supplies  and  services  are  integral  components 
of  the  supply-chain,  which  feeds  production  in  either  government  or  industry.  Without  a 
reliable  flow  of  supplies  and  services,  production  halts,  R&D  slows  and  repairs, 
overhauls,  and  conversions  fall  behind  schedule.  In  interviews  conducted  with 
industry,13  a  history  of  vendor  past-performance  metrics  yield  at  least  three  major 
benefits:  1 )  the  vendor  is  given  a  “report  card”  by  which  to  effect  improvements  to  their 


13  Lockheed  Martin,  interview  by  Roger  Lord,  August  2004.  Northrop  Grumman  Newport  News,  interview 
by  Roger  Lord,  September  2004.  Maersk  Line  Ltd.,  interview  by  Roger  Lord,  September  2004. 
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quality  and  delivery  processes;  2)  the  procuring  activity  has  tangible,  quantifiable,  and 
verifiable  history  of  vendor  performance  if  the  vendor  chooses  to  protest  a  contract 
award;  3)  an  assessment  of  risk  can  be  credibly  assigned  to  the  vendor  prior  to  making 
a  source  selection. 

Past  Performance  Information  Data  Management,  Interview  Question  #4. 

Government:  Active  and  Passive  Approaches 

In  July  2002,  the  Past  Performance  Information  Retrieval  System  (PPIRS) 

became  the  single,  authorized  application  providing  past  performance  reports  to  the 
entire  Federal  acquisition  community  as  a  part  of  the  President’s  e-government 
Integrated  Acquisition  Environment  (IAE)  initiative.14  PPIRS  is  a  web-enabled, 
government-wide  application,  which  provides  timely  and  pertinent  contractor  past- 
performance  information  to  the  Federal  acquisition  community  for  use  in  making  source- 
selection  decisions.  Confidence  in  a  prospective  contractor's  ability  to  satisfactorily 
perform  contract  requirements  is  an  important  source-selection  factor  in  making  best- 
value  decisions  in  the  acquisition  of  goods  and  services.  PPIRS  provides  a  query 
capability  for  authorized  users  to  retrieve  report  card  information  on  a  vendor's  past 
performance.  Federal  regulations  require  that  report  cards  be  completed  annually  by 
customers  during  the  life  of  a  contract.  PPIRS  functions  as  the  central  warehouse  for 
performance  assessment  reports  received  from  the  following  Federal  performance 
information  collection  systems: 

PPIRS  Active  PPI  module(s): 

•  Contractor  Performance  System  (CPS) 

•  Past  Performance  Data  Base  (PPDB) 

•  Past  Performance  Information  Management  System  (PPIMS) 

•  Construction  Contractor  Appraisal  Support  System  (CCASS) 

•  Contractor  Performance  Assessment  Reporting  System  (CPARS) 


14  Under  Secretary  of  Defense,  Acquisition,  Technology  and  Logistics,  Policy  Letter,  5. 
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•  PEDREP  (Product  Data  Reporting  and  Evaluation  Program) 

PPIRS  Passive  PPI  module(s): 

•  Red/Yellow/Green  Program  (U.  S.  Navy) 

•  Automated  Best  Value  System  (DLA) 

Government  access  is  restricted  to  only  those  individuals  who  are  working  on 
source  selections.  Contractors  may  view  only  their  own  data.  Contractor  access  to 
PPIRS  is  gained  through  the  Central  Contractor  Registration  (CCR)  (www.ccr.gov) 
process.  A  contractor  must  be  registered  in  CCR  and  must  have  created  a  Marketing 
Partner  Identification  Number  (MPIN)  in  the  CCR  profile  to  access  their  PPIRS 
information.  PPIRS  is  sponsored  by  the  DOD  E-Business  Office  and  administered  by 
the  Naval  Sea  Logistics  Center  Detachment,  Portsmouth. 

CPARS 

Under  PPIRS,  the  government  activities  interviewed  widely  use  the  Contractor 
Performance  Assessment  Reporting  System  (CPARS)  as  the  centralized  DoD-wide 
system  for  collecting  and  storing  vendor  past  performance  information.  CPARS  is  a 
web-enabled  application  that  collects  and  manages  the  library  of  automated  CPARS. 
CPARS  is  for  UNCLASSIFIED  use  only.  Classified  information  is  not  to  be  entered  into 
this  system.  CPARS  assesses  a  contractor’s  performance  and  provides  a  record,  both 
positive  and  negative,  on  a  given  contract  during  a  specific  period  of  time.  Each 
assessment  is  based  on  objective  facts  and  supported  by  program  and  contract 
management  data,  such  as  cost-performance  reports,  customer  comments,  quality 
reviews,  technical-interchange  meetings,  financial-solvency  assessments, 
construction/production  management  reviews,  contractor-operations  reviews,  functional- 
performance  evaluations,  and  earned-contract  incentives.  CPARS  in  considered  an 
active  PPI  collection  system  because  typically,  the  Program  Manager,  Contracting 
Officers  Representative,  or  other  designated  government  official  will  complete  the 
contractor  evaluation  during  contract  performance  and  upon  completion.  The  report  is 
filed  in  PPIRS. 
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RYG  Program 

The  government  field  contracting  activity  interviewed  also  uses  a  passive  PPI- 
retrieval  system  called  the  Red/Yellow/Green  (RYG)  Program,  for  tracking  only  vendor 
quality  and  delivery.  The  RYG  Program  is  a  closed-loop  system  with  a  central 
receiving  location  which  tracks  quality  and  delivery  by  both  the  vendor’s  Commercial 
and  Government  Entity  (CAGE)  code  and  the  requisition  number  or  purchase  order 
number  awarded  to  the  vendor.  A  color  code  is  assigned  for  both  quality  and  delivery 
by  vendor  CAGE  Code  and  by  each  Federal  Supply  Classification  of  the  product  offered 
by  the  vendor.  The  color  code  assigned  is  based  on  a  computer  algorithm  with  little  or 
no  human  intervention.  The  RYG  Program  appears  to  be  the  closest  correlation  to 
industries’  Scorecard  method.  RYG  is  classified  as  a  passive  system  in  that  it  only 
observes  and  records  PPI  data  as  it  occurs;  yet,  the  CPARS  program  may  be 
considered  an  active  PPI  collection  system  in  that  it  requires  human  intervention  to  feed 
its  data  collection  fields. 


Industry:  Scorecard  Approach 

As  mentioned  earlier,  the  Scorecard  method  of  tracking  vendor  quality  and 
delivery  history  is  employed  by  three  of  the  four  private  firms  interviewed  in  this 
research.  The  Scorecard  method  is  not  web-enabled;  instead,  it  is  information  shared 
on  a  local-area  network  (LAN)  within  the  firm,  or  a  wide-area  network  (WAN)  for  larger 
firms  with  more  than  one  division  within  the  firm.  PPI  information  for  a  particular  vendor 
can  sometimes  be  accessed  by  each  vendor,  or  the  vendor  must  wait  until  a  letter  is 
received  from  the  industry  firm.  PPI  for  a  particular  vendor  is  not  information  that  is 
shared  outside  the  firm.  As  stated  earlier,  both  Scorecard  and  RYG  are  considered 
passive  PPI  collection  systems  in  that  both  Scorecard  and  RYG  monitor  and  record 
vendor  performance  as  performance  occurs.  Recorded  vendor  performance  is  more  or 
less  automatic,  with  PPI  information  fed  to  organized  databases  for  immediate  retrieval. 
Assigned  grades  are  based  on  an  algorithm  scoring  system. 


Scorecard  is  most  effective  and  efficient  when  employed  with  a  “closed  loop” 


supply-chain  in  which  a  central  receiving  area  is  co-located  with  a  quality  control  unit. 


Delivery  and  Quality  Deficiency  Reports  (QDRs)  are  attached  to  the  requisition;  each 
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requisition  is  married  to  the  vendor’s  Commercial  and  Government  Entity  (CAGE)  code. 
Likewise,  the  RYG  program  relies  on  feedback  from  both  the  Shipping  and  Receiving 
unit  and  the  Quality  Control  unit  of  each,  and  is  tied  to  each  vendor. 

Past  Performance  Information  Requirements  for  Requests  for  Proposal 
(RFP),  Interview  Question  #5.  With  little  exception,  government  and  industry  require 
the  same  past  performance  information  from  potential  vendors  prior  to  making  an  award 
decision  as  is  shown  in  Table  8  (pg.  28).  This  past  performance  information 
requirement  is  made  known  via  a  Request  for  Quote  (RFQ)  for  smaller  dollar  value,  less 
complex  requirements  and  a  Request  for  Proposal  (RFP)  for  larger  dollar  value,  more 
complex  requirements. 

Common  required  elements  of  a  potential  vendor’s  past  performance  include:  1) 
recent  performance  (contract  completion  date  of  less  than  three  years),  2)  relevant 
performance  (similar  in  scope  and  complexity)  to  the  anticipated  requirement,  and  3) 
quality  of  performance.''5  Both  government  agencies  and  industry  firms  depend  on  the 
same  kinds  of  past  performance  information.  The  only  exception  to  this  finding  was  the 
Small  Business,  8a  firm  (Industry  Firm  3),  which  performs  little  formal  subcontracting,  or 
outsourcing  to  other  vendors.  This  particular  firm  elects,  instead,  to  perform  and 
maintain  a  wide  range  of  core  competencies  in-house  and  the  use  of  less  formal 
partnering  agreements  if  outsourcing  is  required  to  meet  delivery  deadlines.16 

All  government  activities  and  industry  firms  interviewed  indicated  that  they 
retained  the  right  (in  their  RFP  document)  to  use  past  performance  references  outside 
of  those  provided  by  the  potential  vendor.  This  practice  has  the  obvious  advantage  of 
verifying  a  vendor’s  stated  past  performance,  thus  gaining  a  better  assessment  of  risk. 

In  the  matter  of  Smith  Corp,  B-285358,  21  Aug  2000,  GAO  found  nothing  unreasonable 


15  Office  of  the  Under  Secretary  of  Defense  for  Acquisition,  Technology  &  Logistics,  A  Guide  to  Collection 
and  Use  of  Past  Performance  Information  (Version  3),  Past  Performance  Information  Retrieval  System 
(PPIRS),  May  2003,  6-11. 

16  Opal  Soft,  interview,  August  2004. 
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about  NASA  seeking  information  about  the  offeror’s  performance  on  other  contracts 
relative  [relevant]  to  the  functional  areas  covered  in  the  requirement.17 

Difficulties  Encountered  in  Past  Performance  Information  Collection: 
Government  and  Industry,  Interview  Question  #  6.  This  section  will  be  discussed 
later  in  this  chapter. 

Industry  sources  of  PPI,  Interview  Question  #7.  The  scorecard  method  is  the 
past-performance  collection  and  evaluation  tool  of  choice  within  industry.  As  stated 
earlier  in  this  chapter,  unlike  government  who  makes  PPI  available  for  each  vendor 
across  all  government  agencies  18,  industry  protects  PPI  collected  on  individual  vendors 
in  a  wide-area  network  (WAN)  for  industry  firms  with  more  than  one  geographic  location 
(due  to  mergers,  acquisitions,  and  takeovers),  and  a  local-area  network  (LAN)  for  those 
industry  firms  who  are  smaller  in  size  or  who  have  centralized  their  procurement 
function  to  a  headquarters  location.  The  reasons  for  a  “close-hold”  policy  for  vendor 
PPI  within  industry  is  fairly  obvious.  First,  if  widespread  dissemination  of  a  vendor’s 
documented  PPI  history  is  made  known  to  the  general  public,  it  is  likely  that  legal 
repercussions  would  follow.  All  industry  firms  interviewed  routinely  password-protect 
and  firewall  this  type  of  data19.  Second,  it  could  be  argued  that  industrial  firm  scorecard 
PPI  data  is  proprietary,  and  gained  at  expense  to  the  industry  firm  who  collected  the 
data.  Finally,  if  PPI  scorecard  data  has  been  collected  by  one  potential  customer  prior 
to  pre-award  contract  negotiations  with  the  vendor,  and  the  same  PPI  data  is  not  known 
to  a  second  potential  customer  of  that  same  vendor,  then  the  customer  with  valid  factual 
PPI  data  is  in  a  much  better  position  to  make  a  valid  risk  assessment,  and  perhaps 
negotiate  a  lower  price  than  the  customer  who  has  not  collected  vendor  PPI  data. 


17  Government  Accountability  Office  (GAO),  Smith  Corp,  B-285358,  21  August  2000 

18  Office  of  Federal  Procurement  Policy,  et  al. ,  Best  Practices,  5. 

19  Lockheed  Martin,  Northrop  Grumman  Newport  News,  Maersk  Line,  Ltd.  Interviews  conducted  Aug-Sep 
2004. 
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Summary  of  Findings:  PPI  Collection — Section  2 

This  study  has  shown  thus  far  that  collection  of  past  performance  information 
(PPI)  is  important  to  both  the  Public  Sector  (government)  and  the  Private  Sector 
(industry)  for  the  same  primary  reason.  That  primary  reason  is  risk  mitigation. 

Mitigating  the  risk  of  contract  non-compliance  is  included  in  the  initial  acquisition 
strategy  of  any  procurement,  large  or  small,  for  both  government  and  industry.  The 
data  from  Table  8  suggests  that  reducing  the  risk  of  contract  compliance  involves 
collecting  quality  and  delivery  history.  For  larger  procurements,  cost  control  and 
business  relations  also  become  important  in  the  risk-mitigation  process. 

Where  government  and  industry  part  ways  is  in  the  dissemination  of  the  vendor 
PPI.  As  stated  earlier  in  this  discussion,  Government  is  determined  to  make  vendor  PPI 
retrievable  by  all  government  agencies.  Access  to  Government-collected  PPI  is  firewall 
and  password  protected,  and  typically  accessed  on  a  need-to-know  basis.  In  industry, 
the  dissemination  of  PPI  is  administered  in  a  “close-hold”  need-to-know  basis.  Although 
PPI  is  used  to  estimate  the  level  of  risk  involved  in  selecting  a  particular  vendor  (similar 
to  government),  it  is  arguable  that  PPI  data  held  in  the  private  sector  is  proprietary  in 
nature;  thus,  the  firm  with  better  vendor  PPI  is  the  firm  with  a  competitive  advantage. 

For  the  reasons  described  above,  government  will  likely  continue  to  collect  and 
distribute  vendor  PPI  history  government-wide  while  industry  will  continue  to  collect 
vendor  PPI  history  and  maintain  this  data  close-hold.  At  this  juncture,  the  researcher 
offers  no  recommendations. 
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Table  8:  Summary  of  Findings — PPI  Collection 
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3.  Observations  and  Analysis:  Evaluation  of  Past  Performance 
Information 

There  are  more  commonalities  than  there  are  differences  between  how  the  public 
sector  and  the  private  sector  collect  and  evaluate  contractor  past  performance. 
Surprisingly,  the  entire  acquisition  cycle  of  both  public  and  private  sector  industries 
show  enormous  similarities.  Perhaps  the  most  interesting  section  of  this  research  effort 
was  not  the  collection  of  past  performance  information  (PPI),  but  rather  the  acquisition 
phase  where  vendor  PPI  was  evaluated.  In  this  section  we  will  discuss  and  analyze  the 
commonalities  of  how  government  and  industry  conduct  a  Best-Value  acquisition,  and 
how  vendor  PPI  history  affects  the  outcome.  For  purpose  of  creating  a  baseline,  all 
interviews  assumed  that  the  acquisition  would  exceed  $1M  ($100,000  for  smaller 
companies)  and  employed  a  Best  Value-trade  off  type  approach  acquisition  strategy. 

Active  Members  of  the  Acquisition  Team,  Interview  Question  #8 


Government  and  Industry 

Using  the  assumptions  described  above,  the  investigator  asked  each  activity 
interviewed  to  list  the  typical  active  members  of  their  respective  acquisition  teams.  As 
shown  in  Table  1 1  (pg.  49),  the  procurement  acquisition  team  for  both  government  and 
industry  consists  of  the  Acquisition  Chief/Director,  Procuring  Contracting  Officer, 
Technical  Evaluation  Team,  Past  Performance  Team,  Cost/Price  Analysis  Team,  and 
Engineer/End-user.  Interestingly,  this  is  where  most  of  the  similarities  stop.  Following 
is  a  discussion  and  analysis  of  those  members  of  the  acquisition  process,  which  do  or 
do  not  appear  to  be  major  players  in  routine  large-dollar  acquisitions. 


Legal  counsel.  Interestingly,  only  one  of  the  four  firms  interviewed  indicated 
that  legal  counsel  was  actively  involved  in  the  pre-award  of  a  procurement  action.  One 
of  the  three  firms  indicated  that  they  used  a  legal  advisor  familiar  with  contract  law,  and 
recently  added  a  full-time  legal  counsel.  Nonetheless,  the  data  suggests  that  the 
presence  of  legal  counsel  in  the  pre-award  planning  process  is  conspicuously  absent  in 
industry.  The  researcher  offers  a  discussion/analysis,  as  follows:  1 )  legal  counsel  is 
expensive  to  retain;  2)  the  risk  of  a  disappointed  vendor  protesting  is  almost  non- 
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existent  in  the  private  sector;  3)  senatorial  or  congressional  inquiries  on  behalf  of 
disappointed  vendors  can  normally  be  handled  in-house  at  the  CEO/VP  level. 

However,  once  the  contract  is  awarded,  the  down  side  of  cost-cutting  the  legal  council 
begins  to  catch  up  with  industry.  If  litigation  arises  as  a  result  of  contract  dispute 
between  government  and  industry,  it  is  government  who  prevails  in  two-thirds  of  the 
cases  decided.20 

One  of  the  main  reasons  government  prevails  in  two-thirds  of  all  decided  cases  is 
due  to  the  lack  of  evidence  produced  by  the  plaintiff  (industry).  Typically,  if  a  dispute 
arises  which  cannot  be  disposed  of  by  other  means  (e.g.  Alternative  Dispute  Resolution 
(ADR)  procedures),  the  dispute  must  be  decided  in  either  the  U.S.  Court  of  Federal 
Claims,  or  a  Board  of  Contract  Appeals  (BCA).  Where  government  gains  a  significant 
foothold  is  in  the  very  fact  that  all  government  procurement  activities  maintain  an  on- 
staff  legal  counsel.  This  continuous  legal  coverage  many  times  thwarts  a  potential  legal 
challenge  (e.g.  protest,  claim,  etc.)  by  making  pre-award  recommendations  to  the 
Procuring  Contracting  Officer  (PCO).  Examples  of  good  legal  recommendations  may 
include:  1)  eliminating  a  weak  vendor  proposal  from  the  competitive  range  instead  of 
surprising  the  vendor  on  contract  award  day,  2)  insuring  that  the  government  RFP 
adequately  reflects  all  the  factors  that  the  evaluation  teams  will  be  considering  for 
award,  3)  insuring  that  the  evaluation  teams  fairly  evaluate  all  vendor  proposals  against 
the  criteria  listed  in  the  solicitation,  and  4)  previewing  contracting  officer’s  final  decisions 
(COFD)  for  arbitrary  and  capricious  behavior  prior  to  forwarding  the  COFD  to  the 
vendor. 

With  the  huge  volume  of  contracts  awarded  annually  within  government,  the  legal 
counsel’s  role  as  a  “second  check”  prior  to  release  to  the  open  public  ultimately  pays 
enormous  dividends  in  the  public  sector.  As  an  example,  GAO  denied  a  protest  where 
the  contractor  challenged  the  PCO  regarding  evaluating  factors  listed  in  the  solicitation. 

It  was  discovered  that  the  PCO  had,  in  fact,  considered  only  those  factors  listed  in  the 


20  Lord,  60. 
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solicitation.  GAO  stated,  “Agencies  are  required  to  evaluate  proposals  consistent  with 
the  RFP’s  stated  evaluation  criteria”,  including  considerations  reasonably  and  logically 
encompassed  by  the  stated  factors.21 

Within  industry,  if  a  vendor  submits  a  claim  (in  the  form  of  a  COFD)  which  is  later 
denied,  the  vendor  may  elect  to  litigate.  If  the  vendor  elects  to  litigate,  normally  a 
retained  attorney  is  called  in.  Unfortunately,  the  attorney  enters  the  claim  “in  the 
middle”  and  has  not  been  in  a  position  to  render  good  legal  “steering”  advice  to  the 
vendor  from  the  on-set.  If  the  industry  vendor  and  retained  legal  council  believe  they 
have  a  reasonable  chance  of  prevailing  in  the  courtroom,  the  litigation  decision 
proceeds.  Unfortunately  for  the  industry  vendor,  the  government  PCO  has  been 
receiving  pre-award  and  post-award  legal  “steering”  from  the  start,  whereas  the  industry 
vendor  has  only  been  receiving  post-award  legal  steering;  hence  an  unfavorable  ruling 
is  usually  given,  due  to  the  vendor’s  lack  of  evidence.  Lack  of  evidence  is  the  number 
one  contractor  weakness  in  cases  decided  in  the  Armed  Services  Board  of  Contract 
Appeals.22 

Logistics.  Logistics  planning  is  common  among  government  hardware  system’s 
commands  where  total  ownership  costs  factors  in  reliability,  maintainability,  spares,  and 
support  equipment.  Logistics  planning  is  not  a  large  consideration  within  field 
contracting  commands  where  support  of  these  fielded  systems  take  place.  Industry 
appears  to  use  logistics  planners  less  than  the  government  does.  In  interviews 
conducted  with  industry,  the  explanation  lies  in  the  fact  that  the  engineer/end-user 
(sometimes  called  the  system’s  integrator)  takes  on  this  responsibility  as  part  of  system- 
integration  duties.23 


21  Government  Accountability  Office  (GAO),  Matter  of  Gray  Personnel  Services,  Inc.,  B-285002,  26  June 
2000. 

22  Lord,  101-102. 

23  Lockheed  Martin,  Opalsoft,  Northrop  Grumman  Newport  News,  Maersk  Line  Ltd.,  interviews  conducted 
Aug  -  Sep  2004. 
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Comptroller.  Sporadic  use  of  the  comptroller  in  the  pre-award  planning  in  both 
government  and  industry  acquisition  planning  suggests  that  once  funding  has  been 
identified,  comptrollers  play  a  “hands-off”  role  in  the  contract  pre-award  phase. 
Comptrollers  typically  come  into  play  again  in  the  pre-award  phase  when  incoming  bids, 
quotes,  or  proposals  begin  showing  a  trend  of  higher  than  anticipated  pricing, 
expanding  scope  of  work,  incremental  developments  in  technology  etc.  The  downside 
to  hands-off  comptroller  involvement  can  manifest  itself  in  the  erosion  of  available 
funding  in  favor  of  higher  priority  acquisitions,  unexpected  claims,  or  other  unplanned 
funding  obligations.  However,  the  data  suggests  that  these  are  the  risks  that  both 
public  and  private  sector  are  willing  to  accept. 

Small  Business  Representative.  Within  government  acquisition,  small 
business  (SB)  guidelines  and  goals  continue  to  play  a  huge  role  in  how  contracts  are 
awarded.  Per  the  Federal  Acquisition  Regulation,  all  procurements  with  an  anticipated 
value  between  $2,500  and  $100,000  are  mandated  as  small  business  set-asides. 24 
These  procurements  remain  set-aside  unless  the  contracting  officer  determines  there  is 
not  a  reasonable  expectation  of  obtaining  offers  from  two  or  more  responsible  small 
business  firms.  Presented  another  way,  unless  the  government  PCO  has  determined 
through  adequate  market  research  that  a  small  business  entity  cannot  be  found,  the 
PCO  must  award  the  contract  to  that  small  business.  For  contract  awards  with  an 
anticipated  value  greater  than  $100,000,  the  Small  and  Disadvantaged  Business  Unit 
(SADBU)  representative  may  mandate  a  percentage  of  the  contract  value  to  different 
small  business  segments  (e.g.  Small  Business;  SB,  Veteran  Owned;  SB,  Woman- 
Owned;  HUBZone,  etc.). 

Within  industry,  private  sector  companies  who  receive  government  contract 
awards  must  also  prove  that  they  adhere  to  the  SB  subcontracting  goals  mandated  in 
the  government  contract.  Annual  SB  statistics  are  tracked  and  reported.  Adherence  to 
SB  subcontracting  goals  is  a  sub-factor  monitored  by  government. 


24  Federal  Acquisition  Regulation  (FAR)  Part  6.203  and  FAR  Part  19.502-2(a) 
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QA/QC.  The  use  of  a  QA/QC  rep  during  the  pre-award  of  an  acquisition  phase  is 
also  sporadic  when  factored  across  government  and  industry.  When  a  QA/QC 
representative  is  used  in  the  pre-award,  it  is  industry  that  makes  this  employment,  large 
industry  in  particular.  This  industry  function  makes  obvious  sense.  For  example,  the 
end-product  for  the  two  defense-focused  industry  firms  interviewed  is  D-2  Trident 
missiles  and  nuclear-powered  aircraft  carriers.  In  order  to  remain  competitive,  both 
industry  firms  must  maintain  a  “cradle-to-grave”  quality-control  focus. 

Past  Performance  Data  as  a  factor  in  Source  Selection,  Interview  Question  #9 

When  government  and  industry  forward  a  Request  for  Proposal  (RFP)  to 
interested  offerors,  the  RFP  is  typically  composed  of  a  very  structured  format,  broken 
down  into  sections.  A  Government  RFP  is  structured,  as  follows: 

Section  A — Solicitation/Contract  form 

Section  B — Supplies  or  Services  and  Prices/Costs 

Section  C — Description/Specs/Work  Statement 

Section  D — Packaging  and  Marking 

Section  E — Inspection  and  Acceptance 

Section  F — Deliveries  or  Performance 

Section  G — Contract  Administrative  Data 

Section  H — Special  Contract  Requirements 

Section  I — Contract  Clauses 

Section  J — List  of  Attachments 

Section  K — Representations,  Certifications  and  Other  Statements  of  Offerors 

Section  L — Instructions,  Conditions,  and  Notices  to  Offerors 
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Section  M — Evaluation  Factors  for  Award 


Of  particular  interest  to  the  potential  offeror  is  the  weighting  of  the  individual 
evaluation  factors.  The  PCO  (buyer)  is  responsible  for  constructing  an  RFP  that 
adequately  informs  potential  offerors  (seller)  of  what  the  buyer  needs  (Section  C),  how 
to  prepare  the  proposal  with  the  required  data,  format  and  content  (Section  L),  and  the 
guidelines  as  to  how  each  factor  of  the  proposal  will  be  evaluated  (Section  M). 


FACTOR 

WEIGHTING 

Technical  Approach 

50% 

Sub-Factor  30% 

Sub-Factor  15% 

Sub-Factor  5% 

Past  Performance 

25% 

Corporate  Experience 

15% 

Cost/Price 

10% 

Figure  1:  Typical  Section  M-Factor  weighting  instructions 


The  factors  section  of  Figure  1  is  an  example  of  the  different  parts  of  each 
vendor’s  proposal.  Evaluation  factors  are  typically  weighted  with  the  most  important 
factor  receiving  a  higher  weight.  The  evaluation  factors  and  significant  sub-factors  that 
apply  to  an  acquisition,  as  well  as  those  factors’  relative  importance,  are  within  the 
broad  discretion  of  agency  acquisition  officials.  Although  it  is  not  likely  that  the  RFP 
(which  is  forwarded  to  interested  vendors)  will  disclose  the  detail  shown  in  Figure  1 ,  the 
RFP  will  state,  at  a  minimum,  whether  all  evaluation  factors  (other  than  cost  or  price) 
when  combined  are:  1 )  Significantly  more  important  than  cost  or  price,  2)  Approximately 
equal  to  cost  or  price,  or  3)  Significantly  less  important  than  cost  or  price.  It  is,  however, 
likely  that  the  Source  Selection  Plan  (SSP)  will  contain  this  level  of  detail.  The  SSP  is 
essentially  a  roadmap  or  guideline  established  by  the  PCO  to  evaluate  each  vendor’s 
proposal  against  the  minimum  requirements  set  forth  in  the  RFP. 


As  shown  in  item  nine  of  Table  1 1  (p.  49),  with  the  exception  of  the  SB  firm, 
neither  industry  nor  government  normally  gave  the  past  performance  factor  the  highest 
weight  when  comparing  vendors  in  the  competitive  range.  Technical  capability  was 
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universally  the  factor  with  the  highest  weighting  or  importance.  Therefore,  one  might 
logically  conclude  that  if  the  researcher  were  allowed  to  look  across  government  and 
industry,  each  would  have  an  RFP  with  a  Section  M — Proposal  Evaluation  Criteria  that 
looked  similar  to  Figure  1 .  The  one  industry  member  who  uses  a  total-value  concept 
would  place  an  equal  individual  weight  on  each  factor  considered;  however,  it  is  not 
likely  that  even  such  an  industry  firm  would  give  past  performance  the  highest  weighting 
if  its  respective  acquisition  plan  shifted  to  a  Best-Value  procurement. 25  Put  another 
way,  the  data  suggests  that  within  both  public  and  private  sectors  the  past  performance 
factor  when  placed  in  a  full  and  open  competition  environment  will  receive  less  weight 
than  technical  capability,  but  a  higher  weight  than  cost/price.  However,  within  industry, 
which  typically  does  not  operate  in  a  full  and  open  competition  environment,  it  is 
normally  a  supplier’s  past  performance  which  places  that  supplier  in  the  competitive 
range  along  with  other  preferred  suppliers  instead  of  conducting  an  arduous 
responsibility  determination  which  is  mandated  within  the  public  sector. 

In  summary,  a  vendor  with  a  superior  technical  approach  still  has  a  reasonable 
chance  at  contract  award,  but  it  is  a  thorough  analysis  of  past  performance  which 
distinguishes  between  good  proposal  writers  and  good  performers.26 

Does  Private  Sector  Secretly  Place  Its  Highest  Priority  on  Vendor  Past 
Performance? 

As  suggested  earlier,  not  all  eligible  vendors  are  invited  to  compete  for  private 
sector  contracting  opportunities.  Although  private  sector  solicits  proposals  similar  to 
the  public  sector  in  the  form  of  RFPs,  private  sector  only  forwards  RFP’s  to  approved  “A 
Team”  vendors,  sometimes  called  preferred  vendors.  This  practice  invites  the  argument 
that  private  sector  really  gives  past  performance  the  highest  weight  in  a  best-value  or 
total-value  source  selection.  Further,  those  vendors’  superior  past  performance  is  what 
keeps  the  private  sector  PCO  (buyer)  coming  back  to  the  “A  Team”  vendors  with 


25  Northrop  Grumman  Newport  News  interview,  Sep  2004. 


26  Eric  Stump,  PPI:  Analysis  of  Opinions  from  the  Court  of  Federal  Claims  and  GAO,  Master’s  Thesis 
(Monterey,  CA:  Naval  Post-Graduate  School,  Dec  2001),  18. 
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opportunities  to  provide  quotes,  bids,  or  proposals.  This  observation  will  be  explored  in 
greater  detail  in  interview  question  #1 1  below. 

Has  PPI  Ever  Been  the  Primary  Factor  in  Non-Selection  or  Elimination  from 
the  Competitive  Range,  Interview  Question  #10. 

Most  government  and  industry  procuring  officials  give  technical  approach  a 
higher  weight  than  past  performance  in  Best-Value  source  selections.  Yet,  one 
government  activity  and  two  industry  activities  have  successfully  eliminated  a  potential 
vendor  from  contract  award  based  solely  on  a  history  of  vendor  past  performance.  In  all 
such  cases,  the  activity  interviewee  explained  that  overwhelming  evidence  was  on 
record  to  back  up  their  respective  actions.  However,  non-award  of  a  contract  to  a 
vendor  remaining  in  the  competitive  range  based  on  PPI  alone  is  almost  non-existent; 
therefore,  further  discussion  is  not  warranted. 

Does  Your  Activity  or  Firm  Maintain  a  List  of  Favorite  Go-To  “A  Team” 
Vendors,  in  lieu  of  Full  and  Open  Competition,  Interview  Question  #11. 

The  most  interesting  finding  discovered  by  the  researcher  involved  the  difference 
between  full  and  open  competition  mandated  in  the  public  sector  and  competition 
encouraged  in  the  private  sector. 

Although  there  are  more  similarities  between  public  and  private  sector 
acquisitions  than  differences,  the  researcher  has  uncovered  two  fundamental 
differences  which  stand  out  as  glaring  departures  in  the  typical  >$1 ,000,000  acquisition 
system.  These  differences  include  1 )  Privilege  v.  Right  to  bid,  and  2)  Use  of  full  and 
open  competition. 


ACQUISITION  RESEARCH 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY 
NAVAL  POSTGRADUATE  SCHOOL 


-36- 


1.  Right  v.  Privilege  to  Bid 

The  public  sector  is  mandated  by  the  Federal  Acquisition  Regulation  (FAR)  to 
seek  full  &  open  competition.27  It  is  the  right  of  every  vendor  to  submit  a  quote,  bid  or 
proposal  for  equal  consideration,  regardless  of  past  performance  history.  This  right  is 
extended  to  all  vendors  who  have  not  been  suspended  or  debarred  in  accordance  with 
FAR  Part  9  -  Contractor  Qualifications.  Conversely,  the  private  sector  encourages 
competition  too;  however,  among  their  approved  vendors.  As  shown  in  Table  12  (p. 
50),  question  1 1 ,  all  private  sector  firms  have  approved  vendors.  It  is,  therefore,  a 
defacto  privilege  for  a  vendor  to  offer  a  quote,  bid,  or  proposal  within  the  private  sector. 


Public  Sector 

Private  Sector 

Opportunity  to  offer  a  bid,  quote,  or 
proposal 

Right 

Privilege 

Regulatory  Guidance 

FAR  Part  6  - 

Competition  Reqts 

Mandates 

competition 

UCC  Article  2  -  Sale 
of  Goods  is  silent28 

Table  9:  Opportunity  to  Bid 


Public-Sector  Process 

In  the  Public  Sector,  a  vendor  has  the  right  to  provide  a  bid,  quote,  or  proposal  in 
response  to  needs  of  the  Federal,  State,  and  Local  government.29  Typically  this 
requirement  is  made  known  first  (if  the  anticipated  dollar  value  is  expected  to  exceed 
$25,000)  in  the  form  of  a  synopsis.  A  synopsis  is  a  brief  description  of  the  government 


27  Federal  Acquisition  Regulation  (FAR)  19.201(a)  states:  “It  is  the  policy  of  the  Government  to  provide 
maximum  practical  opportunities  in  its  acquisitions  to  small  business,  veteran-owned  small  business, 
service-disabled  veteran-owned  small  business,  HUBZone  small  business,  small  disadvantaged 
business,  and  women-owned  small  business  concerns....” 

28  Uniform  Commercial  Code  (UCC).  Copyright  1978,  1987,  1988,  1990,  1991,  1992,  1994,  1995,  1998, 
2001  by  The  American  Law  Institute  and  the  National  Conference  of  Commissioners  on  Uniform  State 
Laws:  contains  no  statute  or  requirement  mandating  full  and  open  competition. 

29  Competition  in  Contracting  Act  of  1984  (CICA)  Public  Law  98-369,  sec.  2701,  established  “full  and 
open”  as  a  requirement  in  public  sector  procurements  unless  certain  conditions  exist. 
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requirement.  This  abbreviated  requirement  description  is  made  known  worldwide  on  a 
U.  S.  Government  website  known  as  Federal  Business  Opportunities  (FEDBIZOPPS), 
formerly  known  as  Commerce  Business  Daily  (CBD).  ANY  vendor  may  browse  this 
Government  website  in  search  of  opportunities  within  their  capabilities.  If  interested  in 
bidding,  the  potential  vendor  will  then  contact  the  Procuring  Contracting  Officer  (PCO)  in 
order  to  request  a  solicitation  (RFP).  The  potential  vendor  may  then  elect  to  forward  a 
bid,  quote,  or  proposal.  The  PCO  is  obligated  to  review  for  equal  consideration  all 
timely  bids,  quotes  or  proposals  for  contract  award  (unless  that  vendor  has  been 
suspended  or  debarred  as  outlined  in  FAR  Part  9).  Depending  on  the  value  and 
complexity  of  the  purchase,  the  PCO  may  then  elect  to  make  contract  award.  If  the 
vendor  is  unknown  or  the  proposal  is  complex  or  the  anticipated  contract  value  is  too 
large  to  award  without  prior  approval,  the  PCO  will  elect  to  assign  a  more  formal 
Technical  Evaluation  Board  (TEB)  to  review  the  vendors’  proposals  in  order  to  make  a 
responsibility  determination.30  The  TEB  and  PCO  will  review  each  vendor’s  technical 
approach.  The  PCO  will  also  conduct  a  past-performance  analysis  on  previous  relevant 
contract  awards.  The  PCO  and  Cost  Analyst  will  also  review  the  vendors’  cost 
proposals  as  separate  analyses. 

The  Issue:  Disappointed  Offerors  May  Elect  to  Protest 

All  procurements  will  require  full  &  open  competition  procedures  (unless  officially 
sanctioned  by  the  PCO  in  the  form  of  a  Justification  &  Authorization31  document), 
thereby  giving  all  vendors  a  right  to  participate.  If  the  PCO  fails  to  consider  all  bids, 
quotes  or  proposals  equally,  the  disappointed  vendor(s)  have  the  avenue  to  protest  the 
contract  award.  The  disappointed  vendor  may  then  file  its  protest  directly  to  the 
General  Accounting  Office  (GAO).  The  GAO  has  100  calendar  days  to  render  a 
decision.  If  GAO  denies  the  protest,  the  contract  award  remains  in  effect.  If  GAO 


30  For  purposes  of  simplicity,  this  paper  does  not  refer  to  system  acquisitions  requiring  a  more  formal 
Source  Selection  Evaluation  Board  (SSEB)  and  a  Source  Selection  Advisory  Counsel  (SSAC). 

31  Federal  Acquisition  Regulation  (FAR)  Part  6  -  Competition  Requirements 
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determines  that  the  PCO  violated  the  Competition  in  Contracting  Act  (CICA)  or  a  FAR 
Part,  it  may  elect  to  recommend  one  of  the  following: 

•  Refrain  from  exercising  options  under  the  contract, 

•  Terminate  the  contract, 

•  Re-compete  the  contract, 

•  Issue  a  new  solicitation, 

•  Award  a  contract  consistent  with  statute  and  regulation,  or 

•  Such  other  recommendations  that  GAO  determines  necessary  to  promote 
compliance. 


Evidence  of  difficulties  here  may  be  seen  in  trends  involving  GAO  protests  of  recent 
contract  awards.  Figure  2  depicts  data  on  protests,  with  emphasis  on  protests  involving 
past  performance  as  an  evaluation  criteria,  for  fiscal  years  (FY)  1997  -  1999  and  for  the 
first  three  quarters  of  FY2000. 


FY1997 

FY1998 

FY1999 

FY2000 

(Oct  -  Jun) 

Merit  Protests 

501 

406 

347 

161 

Protests  Sustained 

61 

63 

74 

44 

(Sustainment  Rate  as  %) 

(12%) 

(16%) 

(21%) 

(27%) 

Past  Performance  Protests 

40 

43 

62 

46 

Protests  Sustained 

6 

13 

15 

13 

(Sustainment  Rate  as  %) 

(15%) 

(30%) 

(24%) 

(28%) 

Past  Performance  Protests  as  % 

of  Merit  Protests 

8% 

11% 

18% 

29% 

Sustained  Past  Performance  Protests 

as  %  Sustained  Protests 

10% 

21% 

20% 

30% 

Figure  2.  Summary  of  Recent  GAO  Protests 
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The  data  show  that,  while  the  number  of  merit  protests  (those  not  dismissed  due 
to  procedural  or  substantive  defects)  has  been  declining  over  the  past  four  years,  the 
number  of  past  performance-related  protests  has  been  increasing.  In  1997  past 
performance  protests  constituted  only  8%  of  all  merit  protests,  and  as  of  June  2000  that 
percentage  has  grown  to  29%.  An  upward  trend  is  also  evident  with  the  percentage  of 
past  performance  protests  as  a  percentage  of  sustained  protest.  In  1997  sustained  past 
performance  protests  accounted  for  only  10%  of  the  overall  sustained  protests,  and  by 
June  of  2000  that  percentage  had  risen  to  30%.  32 

Interestingly,  as  government  has  moved  to  adopt  industry  best  practices  for 
source  selections  based  on  vendor  past  performance,  industry  has  responded  with 
increased  protests.  This  discovery  that  industry  suspects  unfair  source  selection 
practices  only  reinforces  the  need  for  government  to  maintain  a  robust  PPI  collection 
system  for  retrieving  current  and  relevant  vendor  performance  history  and  that 
government  exercise  due  diligence  in  their  responsibilities  as  source  selection  officials 
to  apply  evaluation  criteria  consistently  across  all  vendor  bids,  quotes  or  proposals  in 
accordance  with  evaluation  criteria  set  forth  in  each  solicitation.  This  also  emphasizes 
the  notion  that  government  is  indeed  different  than  industry,33 further  that  the  privilege  to 
bid  (private  sector)  versus  the  right  to  bid  (public  sector)  does  not  lend  to  a  smooth  and 
seamless  adoption  of  industry  best  practices  under  current  public  laws,  statutes,  and 
regulations.  Again,  this  reinforces  the  argument(s)  to  collect  and  evaluate  vendor  PPI 
in  a  manner  that  is  ethical,  verifiable,  and  absent  any  perception  of  impropriety.  It 
should  also  be  noted  that  the  U.  S.  Court  of  Federal  Claims  also  has  jurisdiction  over 
both  pre-  and  post-award  protests  per  the  Administrative  Dispute  Resolution  Act  of 
1996.  In  short,  the  government  PCO  is  required  to  give  equal  consideration  to  all 
incoming  bids,  quotes  or  proposals  until  such  time  that  a  determination  is  made  and 


32  Keith  Snider  and  Mark  Walkner.  Best  Practices  and  Protests:  Toward  Effective  Use  Of  Past 
Performance  As  a  Criterion  in  Source  Selections.  Journal  of  Public  Procurement,  Vol  1 ,  Issue  1 . 
(PrAcademics  Press,  2001),  99  -  101 

33  Ibid.,  100. 
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documented  in  the  contract  file  that  the  vendors  bid,  quote,  or  proposal  is  determined  to 
be  non-responsive  to  the  solicitation  and/or  the  vendor  is  determined  to  be  non- 
responsible. 

Private-Sector  Process 

The  private  sector  is  guided  by  the  Uniform  Commercial  Code  (UCC)  in  routine 
business  practices;  yet  UCC  Article  2,  “Sale  of  Goods,”  is  silent  regarding  competition 
when  soliciting  bid,  quotes,  or  proposals.  What  is  decidedly  different  is  that  the  private 
sector  is  not  mandated  to  seek  full  &  open  competition.  Disappointed  vendors  have  no 
legal  recourse  with  GAO  or  the  U.  S.  Court  of  Federal  Claims  if  they  are  not  asked  by  a 
private  sector  firm  to  provide  a  bid,  quote,  or  proposal,  or  if  their  bid,  quote  or  proposal 
was  not  selected  for  contract  award.  The  only  viable  alternative  for  the  disappointed 
vendor  is  to  call  their  respective  Congressman  or  Senator.  The  research  data  suggests 
that  congressional  inquiries  are  more  likely  to  be  directed  to  larger  firms  in  the  private 
sector  than  smaller  firms.  Research  interviews  conducted  as  part  of  this  study  indicate 
that  answering  congressional  and  senatorial  inquiries  is  taken  seriously  and  handled 
professionally  within  the  private  sector,  yet  they  do  not  have  the  disruptive  nature  that  a 
GAO  Protest  has  in  the  public  sector.  Hence,  it  is  not  likely  that  a  congressional  inquest 
will  delay  a  contract  award  or  halt  contract  performance  in  the  private  sector.34 


34  Northrop  Grumman,  Lockheed  Martin  interviews  conducted  Aug  &  Sep  2004. 
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Requirement 

Identified 

Market  Research 

Responsibility 

Determination 

Competitive 

Range 

Determination 

Best  Value 

Contract 

Award 

Government 

•  Synopsis  world-wide 

•  Pre-Solicitation 

Conference 

•  Pre-Award  Survey 

•  Site  visits 

•  Tech  Approach 

•  Past  Perf.  Eval 

•  Cost  Analysis 

Contract 

Award 

Industry 

(With  preferred 
vendor  support) 

•  Tech  Approach 

•  Past  Perf. 

•  Eval  Cost  Analysis 

Contract 

Award 

Industry 

(Mature  vendor 
relations,  Lean,  JIT 
etc.) 

Contract 

Award 

Table  10.  Responsibility  Determination  Comparison:  Public  v.  Private  Sector 


2.  Full  and  Open  Competition:  a  Socioeconomic  Process 

The  other  striking  difference  between  public  sector  and  private  sector 
procurement  processes  deals  with  full  and  open  competition.  Government  acquisition  is 
a  socio-economic  process.  Its  number-one  priority  is  supporting  socioeconomic  goals 
mandated  by  public  law.  After  priority  number  one  is  met,  priority  number  two  is  the 
acquisition  of  the  right  supplies,  services,  construction,  R&D,  etc.  at  the  right  time  and  at 
the  right  price. 

Government 

Public  sector  is  mandated  to  compete  all  requirements  using  full  and  open 
competition  procedures  whenever  feasible.35  What  is  less  obvious  to  an  observer  is  the 
burden  of  proof  placed  on  the  PCO  to  prove  competition  does  or  does  not  exist  OR  that 


35  Federal  Acquisition  Regulation  (FAR)  Part  6.101  -  Full  and  Open  Competition  Policy 
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it  is  in  the  best  interest  of  the  government  not  to  compete  the  requirement.36  This  public- 
sector  process  of  bypassing  the  full  and  open  competition  requirement  is  exacting  and 
time  consuming.  Yet,  industry,  as  discussed  previously,  has  no  competition 
requirement. 

Within  Government,  unless  exceptions  to  full  and  open  competition  are 
approved,  government  is  obligated  to  proceed  with  market  research  and  a  worldwide 
search  (synopsis)  in  order  to  determine  if  competition  exists  (as  shown  in  Table  10). 
Once  this  step  is  performed,  an  RFP  is  forwarded  to  interested  vendors.  The  vendor 
returns  to  the  PCO  a  timely  quote,  bid,  or  proposal.  If  a  vendor  is  unknown,  the  PCO 
may  elect  to  perform  a  pre-award  survey  to  determine  vendor  responsibility.  A  pre¬ 
award  survey  is  normally  required  only  when  the  information  on  hand  regarding  a 
particular  vendor  is  not  sufficient  to  make  a  determination  regarding  responsibility. 
Typical  pre-award  survey  areas  may  include  a  detailed  analysis  of  the  firm’s:  1) 
technical  capabilities,  2)  production  processes,  3)  quality-control  procedures,  4) 
financial  health  of  the  vendor,  and  5)  accounting  practices.  Additional  procedures  for 
“bringing  on”  new  vendors  may  include  a  pre-solicitation  conference  (if  the  requirement 
is  new  to  both  government  and  industry),  site  visits  (if  the  requirement  is  new  to  a  group 
of  vendors),  and  other  methods  of  determining  vendor  responsibility.  Making  a 
responsibility  determination  on  a  new  vendor  is  costly,  time  consuming  and  extends  the 
procurement  action  lead  time  (PALT)  of  a  requirement  by  weeks,  months  or  perhaps 
years. 


Industry 

Similar  to  government,  once  a  requirement  and  funding  is  identified  by  an 
industry,  a  solicitation  is  forwarded  to  interested  vendors;  yet,  unlike  government,  the 
new  requirement  normally  is  forwarded  to  a  pre-approved  vendor  list,  or  as  some 
industry  counterparts  call  it:  an  “A  Team.”  A  world-wide  search  for  new  vendors  is  not 


36  Federal  Acquisition  Regulation  (FAR)  Part  6.302:  1  through  7 
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performed  unless  it  makes  sense  to  do  so.37  Full  and  open  competition  is  not  normally 
used.  This  unique  list  of  vendors  includes  only  the  vendors  who  will  normally  receive  an 
RFP  from  an  industry  procurement  department.  The  “A  Team”  vendor  has  a  proven 
track  record  of  high  quality  and  on-time  deliveries  based  on  months,  years,  or  decades 
of  reliable  performance.  Hence,  the  procedures  for  making  a  responsibility 
determination  are  eliminated  in  the  private  sector.  The  data  reveals  that  all  private 
industry  firms  maintain  a  list  of  reliable  “A  Team”  vendors  in  one  form  or  another. 38  For 
more  information,  see  also  Table  12  question  1 1.  The  efficiency  gained  by  using  an 
approved,  reliable  vendor  list  is  exponential,  as  shown  in  Figure  3.  Depending  on  the 
nature  and  urgency  of  the  requirement,  industry  may  elect  to  either  review  bids,  quotes, 
or  proposals  from  it’s  “A  Team”  vendors  or  proceed  directly  to  contract  award.  This 
decision  to  go  directly  to  contract  award  is  strictly  the  prerogative  of  the  industry 
procurement  official.  Again,  purchase  agreements,  partnerships,  or  strategic  alliances 
between  industry  and  their  ”A  Team”  vendors  may  last  months,  years,  or  decades  and 
beyond. 

Perhaps  the  government  contract  arrangements  closest  to  these  long-term 
private-sector  relationships  are  Indefinite  Delivery  Type  Contracts  and  Performance- 
Based  Contract  vehicles;  although  these  are  both  typically  re-competed  after  only  five 
years.  Although  government  is  headed  in  the  right  direction  towards  improved 
efficiency,  the  typical  Indefinite  Delivery  or  Performance-Based  Contract  vehicle  has  a 
relatively  short  life-span  and  requires  that  all  like-requirements  go  only  to  that  particular 
contract  awardee. 

In  the  government  model,  a  typical  requirement,  based  on  the  researcher’s 
experience,  could  realistically  reach  a  210-day  PALT,  as  shown  in  Figure  3. 


37  Northrop  Grumman,  interview  Sep  2004. 

38  Lockheed  Martin,  Opalsoft,  Northrop  Grumman  Newport  News,  Maersk  Line  Ltd.,  interviews  Aug  -  Sep 
2004. 
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In  the  industry  procurement  model,  the  following  steps  or  blocks  are  effectively 
eliminated:  1)  Synopsis,  2)  Responsibility  Determination,  and  3)  a  separate  Best-Value 
Analysis  which  is  combined  into  one  heading  under  Total  Value  or  Best  Value.  The 
researcher  has  estimated  a  PALT  from  a  similar  procurement  action  (based  on  private 
sector  interviews)  under  similar  urgency  conditions  to  reach  contract  award  in  75  days, 
or  about  one  third  of  the  time  of  the  government  model,  as  shown  in  Figure  3  below: 


Government  Model: 


Responsibility1 

Determination: 

-  Tech  Eva  I 

-  Fast  F'  erf  Eva  I 

-  Cost  Analysis 

-  Frs-Award 

Surveys 

-  Cite  Visits 


Competitive 

Range 

Determ¬ 

ination 


Best 

Value 

Analysis 


Contract 

Award 


Industry  Model: 


Requirement 

RFG/ 

Assessment 

CDuiid  or  huyi 

► 

nrn 

1 


Funding 
FjEqt  Doc 


Best /Total 

value 
Analysis 
- 1  ecn 
Past  F'erf 
-  Cost 

► 

Contract 

Av-ad 

Government  timeline: 

30  days  15  days  45  days  60  days  30  days  30  days  =  210  days 

Industry  timeline: 

15  days  30  days  30  days  =  75  days 

Figure  3:  Government  v.  Industry  Procurement  Action  Lead  Time  (PALT)  Comparison 

It  should  be  noted  that  industry,  similar  to  government,  must  occasionally  follow 
the  full  acquisition  process  when  a  new  requirement  justifies  the  search  for  a  new 
supplier(s).  The  search  for  new  suppliers  is  seen  as  rare  by  industry  acquisition 
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specialists  and  only  executed  when  it  makes  sense  to  do  so. 39  If  government  is  to 
continue  evolving  and  incorporating  industry  best  practices,  improving  PALT  by 
eliminating  burdensome  acquisition  processes  is  a  viable  area  for  consideration.  These 
recommendations  will  be  explored  in  the  following  chapter. 

RFQ/RFP  Evaluation  Factors,  Interview  Question  #12.  The  common  factors 
evaluated  by  both  government  and  industry,  as  shown  in  Table  12  (pg.50),  within  the 
confines  of  past  performance  are:  1)  Relevancy  (how  similar  is  referenced  work  to  the 
scope  of  work  outlined  in  the  requirement?);  2)  Currency  (was  work  performed  with  a 
completion  within  the  past  three  years?);  3)  Quality  (how  well  did  the  vendor  perform  as 
stated  on  evaluations,  i.e.  CPARS?). 

In  other  words,  when  a  vendor  is  asked  to  provide  past-performance  history  in 
response  to  Section  L — Proposal  Requirements  of  the  RFP,  the  common  factors 
considered  by  government  or  industry  past-performance  evaluation  boards  are  those 
stated  above.  Vendor  experience  and  other  factors  are  considered  but  not  consistently 
enough  to  provide  comment. 

Research  of  Named  References 

Once  past-performance  references  were  identified  in  the  potential  vendor’s 
proposal,  the  past-performance  evaluation  teams  (both  government  and  industry) 
favored  calling  (performing  telephone  interviews  with)  those  firms  who  were  referenced. 
Questionnaires  from  the  requiring  activity  to  each  prospective  vendor  were  also  used; 
these  were  used  primarily  by  government  procurement  officials  and  less  so  by 
procurement  officials  within  industry. 

Research  of  Other-Than-Named  References 

The  government  appears  to  be  more  aggressive,  as  shown  in  Table  12,  item  12, 
than  industry  in  regards  to  conducting  past-performance  checks  using  sources  other 


39  Lockheed  Martin,  Opalsoft,  Northrop  Grumman  Newport  News,  Maersk  Line  Ltd.,  interviews  conducted 
Aug  -  Sep  2004. 
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than  those  listed  in  the  past-performance  section  of  a  vendor’s  proposal.  The  use  of  1) 
other  end  users,  2)  other  customers,  3)  passive  PPI  (i.e.  QDRs),  and  4)  pre-award 
surveys  are  more  common  in  the  public  sector  than  in  the  private,  and  are  all  within  the 
governments’  rights.40 

Two  problems  have  been  discovered  during  these  research  efforts  which 
necessitate  government  procurement  officials  to  search  for  past-performance 
information  outside  of  what  is  offered  by  the  vendor  in  his/her  proposal: 

Lack  of  PPI  Data  in  Government  Agency  Databases: 

Based  on  the  researcher’s  experience,  and  verified  by  research  interviews,  PPI 
located  in  the  PPIRS/CPARS  database  is  not  always  available  on  all  vendors.41  Either 
due  to  the  neglect  of  the  procurement  activity  to  make  timely  CPARS  assessments,  or 
the  contract  value  which  may  not  warrant  CPARS  surveillance,  PPI  data  is  simply  not 
available  much  of  the  time.  This  problem  is  more  pronounced  in  field  contracting 
activities  where  there  are  multiple  vendors  and  the  contract  values  are  typically  lower 
than  contract  values  at  larger  system  commands.  Missing  PPI  is  less  of  an  issue  in 
hardware  system  commands  where  the  number  of  vendors  capable  of  producing  major 
weapon  systems  is  limited  to  a  handful  of  well-known  vendors. 

CPARS  Grade  Inflation: 

So  why  do  the  larger  hardware  system  commands  also  reach  out  for  PPI  outside 
the  data  found  in  PPIRS/CPARS  and  past-performance  history  offered  by  the  potential 
vendor  is  his/her  proposal?  Surprisingly,  the  answer  lies  in  the  fact  the  Program 
Managers  of  major  systems  routinely  inflate  grades  given  to  vendors  on  mid-term 
CPARS  evaluations.  The  driver  behind  inflated  CPARS  interim  reports  lies  in  perceived 
program  risk,  and  the  potential  for  a  reduction  in  funding.  The  function  of  the  Program 
Manager  is  to  field  a  weapons  system:  1 )  on  time,  2)  under  budget,  and  3)  according  to 


40  U.  S.  Court  of  Federal  Claims,  Smith  Corporation ,  B-285358,  21  August  2000. 

41  Lockheed  Martin,  Opalsoft,  Northrop  Grumman  Newport  News,  Maersk  Line  Ltd.,  interviews  Aug  -  Sep 
2004. 
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Key  Performance  Parameters  (KPPs).  If  there  is  a  perception  that  a  program  (weapons 
system)  is  in  jeopardy  or  at  risk  of  non-performance  perhaps  brought  about  by  attention 
gained  in  giving  a  “bad  CPARS  interim  report,”  congress  may,  and  has  indeed  been 
known  to,  “pull  funds”  on  weapons  system  programs,  thereby  killing  the  program. 
Program  Managers  are  trained  to  protect  their  program,  and  in  doing  so,  the  jobs  it 
provides;  therefore,  a  grade-inflation  problem  persists;  and,  government  procurement 
officials  will  continue  to  state  their  intention  to  seek  PPI  outside  of  agencies’ 
database(s). 
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Research 

Govt 

Govt 

Govt 

Govt 

Industry 

Industry 

Industry 

Industry 

Question 

Acty  1 

Acty  2 

Acty  3 

Acty  4 

Firm  1 

Firm  2 

Firm  3 

Firm  4 

NAVAIR 

NAVSEA 

FISC  / 

TACOM 

LOCK¬ 

NORTH¬ 

OPAL 

MAERSK 

NNSY 

HEED 

ROP 

SOFT 

LINE 

Section  3a 

8.  Active  mbrs  of 
your  Acqn  Team: 

Evaluation 
of  Past 

-  Acqn  Chief 
/Director 

♦ 

♦ 

♦ 

♦ 

♦ 

♦ 

♦ 

♦ 

Performance 

-  PCO 

♦ 

♦ 

♦ 

♦ 

♦ 

♦ 

♦ 

♦ 

Data 

-  Legal  counsel 

♦ 

♦ 

♦ 

♦ 

♦ 

-  Technical 
Evaluation  Tm 

♦ 

♦ 

♦ 

♦ 

♦ 

♦ 

♦ 

♦ 

-  Past 

Performance  Tm 

♦ 

♦ 

♦ 

♦ 

♦ 

♦ 

♦ 

♦ 

-  Cost/Price 
Analysis  Tm 

♦ 

♦ 

♦ 

♦ 

♦ 

♦ 

♦ 

♦ 

-  Engineer/  End- 
user 

♦ 

♦ 

♦ 

♦ 

♦ 

♦ 

♦ 

♦ 

-  Logistics 

♦ 

♦ 

♦ 

♦ 

-  Comptroller 

♦ 

♦ 

♦ 

-  Sm  Bus  Rep 

♦ 

♦ 

♦ 

♦ 

♦ 

♦ 

-  QA/QC 

♦ 

♦ 

9.  PPI  Data  as  a 
factor  in  source 
selection: 

-  Ranking 

4 

4 

4 

4 

4 

4 

4 

4 

Past  Perf. 

3 

3 

3 

3 

3 

3 

3 

3 

Highest  weight  =  4 

2 

2 

2 

2 

2 

2 

2 

2 

lowest  =  1 

1 

1 

1 

1 

1 

1 

1 

1 

Total 

Value 

10.  Has  PPI  ever 
been  the  primary 
factor  in  non¬ 
selection  or 

Yes 

elimin.  from 
competitive  rg? 

(rarely) 

No 

No 

No 

No 

Yes 

Yes 

No 

-  Legally 

challenged  by  Ktr? 

No 

N/A 

N/A 

N/A 

N/A 

No 

No 

N/A 

-  Sustained  in 
favor  of  Ktr? 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

Table  11.  Section  3a:  Summary  of  Findings — PPI  Evaluation 
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Research 
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Govt 

Acty  1 

NAVAIR 

Govt 
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NAVSEA 

Govt 
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FISC  / 

NNSY 

Govt 

Acty  4 

TACOM 

Industry 

Firm  1 

LOCK¬ 

HEED 

Industry 

Firm  2 

NORTH¬ 

ROP 

Industry 

Firm  3 

OPAL 

SOFT 

Industry 

Firm  4 

MAERSK 

LINE 

Section  3b 

Evaluation  of 
Past 

Performance 

data 

11.  List  of 
favorite  go-to 
firms  in  lieu 
of 

competition? 

♦ 

♦ 

♦ 

♦ 

12.  RFP/bid 

evaluation 

factors: 

-  Relevancy 

♦ 

♦ 

♦ 

♦ 

♦ 

♦ 

♦ 

♦ 

-  Currency  < 

3yrs 

♦ 

♦ 

♦ 

♦ 

♦ 

♦ 

♦ 

♦ 

-  Quality 

♦ 

♦ 

♦ 

♦ 

♦ 

♦ 

♦ 

♦ 

-  Experience 

♦ 

♦ 

♦ 

♦ 

-  Other  factors 

♦ 

♦ 

a.  Research  of 

named 

references: 

-  Telephone 

♦ 

♦ 

♦ 

♦ 

♦ 

♦ 

♦ 

Questionnaires 

♦ 

♦ 

♦ 

♦ 

♦ 

b.  Research  of 
other-than- 
named 
references: 

-  End-Users 

♦ 

♦ 

♦ 

♦ 

♦ 

♦ 

-  Other 

Customers 

♦ 

♦ 

♦ 

♦ 

♦ 

-  Passive  PPI 
(i.e.  QDRs) 

♦ 

♦ 

♦ 

♦ 

♦ 

♦ 

♦ 

-  Pre-award 
surveys 

♦ 

♦ 

♦ 

♦ 

♦ 

♦ 

♦ 

Table  12.  Section  3b:  Summary  of  Findings — PPI  Evaluation 


Summary  of  Findings 

As  stated  in  the  preceding  paragraphs,  the  differences  between  the  way  Public 
Sector  (government)  and  Private  Sector  (industry)  evaluate  vendor  bids,  quotes,  and 
proposals  and  make  contract  awards  is  many  times  blurred.  Both  government  and 
industry  use  a  similar  acquisition  team,  although  government  contracting  activities  more 
aggressively  use  legal  counsel  in  the  pre-award  phase;  both  government  and  industry 
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tend  to  place  the  level  of  importance  of  PPI  somewhere  between  technical  capability 
and  cost/price;  and  both  appear  to  evaluate  potential  vendor’s  PPI  in  much  the  same 
manner.  Where  government  and  industry  differ  is  in  the  level  of  vendor  participation. 
Government  is  mandated  to  entertain  full  and  open  competition,  including  unknown 
vendors,  and  to  treat  all  equally.  Industry,  on  the  other  hand,  needs  only  solicit  quotes, 
bids  or  proposals  from  their  approved  vendor  list  (preferred  vendor),  thereby  effectively 
eliminating  the  need  for  a  lengthy  responsibility  (capability)  determination. 

Therefore,  vendors  have  a  right  to  provide  a  bid,  quote,  or  proposal  in  response 
to  a  government  RFP,  whereas  it  is  a  privilege  to  be  invited  to  provide  a  bid,  quote,  or 
proposal  in  response  to  an  industry  RFP.  Government  procurement  officials  are  under 
the  scrutiny  of  the  public,  vendor  rights  (protests  or  claims),  the  General  Accounting 
Office  (GAO)  and  the  U.  S.  Court  of  Federal  Claims.  Industry  is  also  under  the  scrutiny 
of  the  public,  yet  to  a  lesser  degree,  and  vendor  recourse  to  an  unfavorable  Industry 
procurement  decision  is  primarily  via  Congressional  or  Senatorial  inquires.  Therefore, 
one  could  argue  that  industry  actually  places  vendor  past  performance  above  technical 
capability/quality  or  cost.  If  industry  keeps  reliable  history  on  their  vendors,  it  is 
effectively  using  past  performance  as  its  most  important,  or  highest-weighted,  source- 
selection  criteria. 

Finally,  both  government  and  industry  tend  to  evaluate  vendor  past-performance 
history  using  the  same  basic  criteria:  1 )  relevancy,  2)  currency,  and  3)  quality  of 
performance.  Government  procurement  officials  tend  to  search  out  PPI  sources  outside 
of  vendor-offered  PPI  references  and  internal  PPI  databases  due  to:  1)  missing  vendor 
PPI  data  in  government-wide  PPI  databases,  and  2)  PPI  grade  inflation  on  interim 
vendor  performance  reports.  The  data  also  suggest  that  PPI  grade  inflation  appears  to 
be  centrally  linked  to  vendors  for  major  weapon  systems  where  congressional  scrutiny 
is  prevalent. 
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E.  ANALYSIS  OF  RESEARCH  QUESTIONS 


The  two  basic  research  questions  under  study  were  as  follows: 

1.  What  are  the  principal  issues  involved  in  using  PPI  in  the  source-selection 
process? 

The  principle  issues  uncovered  in  using  past-performance  information  (PPI)  in 
the  source-selection  process  are  as  follows: 

a.  A  robust  PPI  infrastructure  is  required  to  justify  source  selections  to  other  than 
the  lowest-priced  technically  acceptable  (LPTA)  offeror.  Past  performance  history  is  a 
significant  source-selection  factor  in  awarding  contracts  to  other  than  the  lowest-priced 
offeror  in  both  government  and  industry. 

GOVERNMENT.  Within  government,  a  robust,  web-enabled  PPI  infrastructure 
accessible  across  all  government  agencies  is  essential.  If  Government  procurement 
officials  are  to  successfully  carry  out  the  procurement  policy  objectives  of  achieving  pre¬ 
determined  socioeconomic  goals42  and  successfully  executing  best-value  procurements 
to  other  than  the  LPTA  offeror,  relevant,  current,  and  quality  PPI  vendor  history  must  be 
readily  accessible.  The  evaluation  and  analysis  of  verifiable  and  quantifiable  data  in 
both  the  technical  capability  and  past-performance  history  source-selection  factors  must 
be  documented  in  the  contract  file  in  order  to  avoid  sustainable  protests  by  disappointed 
offerors. 

INDUSTRY.  The  research  data  suggests  that  industry,  too,  uses  PPI  as  a  factor 
in  source  selections.  Yet,  industry’s  emphasis  on  vendor  past  performance  takes  a 
somewhat  different  role.  In  “Raytheon’s  Case  for  Action”  the  CEO  of  the  organization 
recognized  the  pivotal  role  of  supply  chain  management.  He  also  noted  the  fact  that 
poor  supplier  performance  was  impacting  key  programs;  likewise,  new  suppliers  were 
being  added  without  justification  when  existing  suppliers  had  the  capability  and  capacity 


42  Federal  Acquisition  Regulation  (FAR)  Part  1.102  -  Statement  of  guiding  principles  for  the  Federal 
Acquisition  System. 
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to  meet  requirements.43  Raytheon’s  analysis  revealed  that  significant  benefits  were 
obtainable  through  leveraging  effective  supplier  management,  proactive  supplier 
development,  and  by  strategically  focusing  their  resources  on  fewer  suppliers.  The 
case  to  be  made  is  that  industry  uses  PPI  first  to  eliminate  poor  suppliers  entirely  and 
secondly  to  form  short-term  contracts  or  longer-term  alliances  with  those  suppliers  who 
remain. 

b.  PPI  is  generally  used  as  a  risk  assessment  and  to  separate  the  good  proposal 
writers  from  the  good  vendors.  This  observation  is  geared  more  toward  government 
than  towards  industry.  When  two  or  more  vendors  are  compared,  all  of  whom  already 
have  acceptable  technical  proposals,  only  then  does  past  performance  appear  to 
become  a  significant  factor  in  the  source-selection  process.  Up  to  this  point,  the 
question  has  been  whether  the  vendor  understands  the  scope  of  work,  has  answered 
the  solicitation  with  a  proposal  which  will  satisfy  the  activity’s  or  firm’s  minimum 
requirement(s),  and  has  the  best  price  relative  to  the  value  of  the  proposed  product. 
Only  now  does  the  question  shift  to  a  vendor’s  quality  and  delivery  history.  Will  the 
prospective  vendor  deliver  a  quality  product  on  time?  This  is  where  the  vendor  with  the 
best  PPI  track  record  (in  a  Best-Value  source  selection)  wins  the  contract  award;  yet,  it 
will  only  win  the  contract  award  if  its  respective  technical  proposal  is  at  least  acceptable. 

c.  PPI  is  typically  never  used  as  a  stand-alone  factor  for  non-contract  award  in  a 
full  and  open  competition  environment  (Govt)  or  when  using  a  preferred  vendor  list 
(Industry).  Surprisingly,  the  PPI  source-selection  factor  is  never  the  highest-weighted 
factor  in  source  selections  within  government,  and  is  seldom  used  as  the  highest- 
weighted  factor  within  industry  amongst  its  A-Team  vendors.  In  the  rare  case  where 
PPI  was  successfully  used  as  the  sole  discriminator  for  non-award,  the  industry 
procurement  officials  were  able  to  respond  to  senatorial  inquiries  with  convincing, 
quantifiable  PPI  history  to  back  up  their  decisions. 


43  Shirley  Patterson,  “Supply  Base  Optimization  and  Integrated  Supply  Chain  Management”  Contract 
Management  volume  45,  issue  1  (January  2005):  25. 
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d.  PPI  can  only  be  defendable  if  made  quantifiable.  As  stated  above,  PPI 
history  is  only  useful  and  defendable  if  made  quantifiable.  For  this  reason,  grades 
(either  numerical,  adjectival,  or  colorful)  have  become  an  integral  part  of  PPIRS 
(government)  and  Scorecard  (industry). 

e.  An  active  (or  passive)  PPI  collection  system  continues  to  be  burdensome,  yet 
useful.  Within  both  government  and  industry,  the  use  of  vendor  past-performance 
information  (PPI)  history  in  the  source-selection  process  involves  significant  government 
or  industry  resources.  The  use  of  these  resources,  or  more  specifically  the  use  of 
human  capital,  is  required  to  collect  and  maintain  reliable  PPI  databases.  This  process 
is  burdensome,  regardless  as  to  whether  it  is  a  web-enabled,  globally  accessible 
program  or  a  locally  maintained  manual  process.  With  this  said,  when 
government/industry  can  produce  a  quantifiable  history  of  vendor  PPI  (e.g.  number  of 
late  deliveries,  percentage  of  quality  rejects  over  a  period  of  time),  an  adjectival  grade  of 
high  risk,  medium  risk,  or  low  risk  will  be  easier  to  obtain  and  defend. 

2.  How  might  an  assessment  of  industry  models  of  past-performance  evaluation 
assist  in  improving  the  DoD’s  use  of  past  performance  as  an  evaluation  criterion? 

Although  government  activities  fell  in  line  with  one  another  regarding  acquisition 
processes  and  procedures,  industries  varied  somewhat  in  their  acquisition  process 
methodology  and  their  methodology  for  collecting  and  evaluating  vendor  past 
performance.  Therefore,  the  following  tables  represent  an  aggregate  list  of  findings  and 
do  not  reflect  any  one  particular  government  activity  or  industry  firm. 
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PPI  Collection: 

Government  Model 

Industry  Model 

PPI  Collection  Sys. 

PPIRS/CPARS  (Active) 

SCORECARD  (Passive) 

-  Quality 

-  Quality 

-  Delivery 

-  Delivery 

-  Cost  Control 

Manual  sys  for  tracking  Cost 

-  Business  Relations 

Manual  System  for  tracking 
Business  relations 

PPIRS/RYG  (Passive) 

-  Quality 

-  Delivery 

Class  of  Collection  Sys. 

Active 

Passive 

Prgm  Mgrs  actively  engaged 
in  grading  interim  and  final 
vendor  performance 

Vendor  performance  is 
monitored  by  a 

shipping/receiving  system  and 
QC  personnel 

Passive 

Vendor  performance  is 
monitored  by  a 

shipping/receiving  system  and 
QC  personnel 

Table  13.  Summary  of  Government  v.  Industry  PPI  Collection  Systems 
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PPI  Evaluation: 

Government  Model 

Industry  Model 

Key  Elements 

Evaluated 

Quality 

Quality 

Delivery 

Delivery 

Cost  Control 

Cost  Control  (manual) 

Business  Relations 

Business  Relations  (manual) 

Priorities 

1)  Socio-economic  objectives, 

2)  Cost,  Schedule, 

Performance 

1)  Profit  &  shareholder 
interests,  2)  Reputation  based 
on  quality 

Competition 

-  Full  and  Open  competition 
(FAR  Part  6  and  CICA  of 

1984) 

-  No  competition  requirement 
(UCC  Article  2  is  silent) 

Invitation  to  provide 
bid,  quote,  proposal 

Right 

Privilege 

Most  important  source 
selection  factor 

Technical  Capability 

Past  Performance/Use  of  pre¬ 
approved  vendors 

Best  value  source 
selection 

Responsibility  Determination: 

1.  Technical  Capability 

2.  Past  Performance 

3.  Cost/Price 

After  a  lengthy  evaluation  of 
proposals,  only  the  offerors 
whose  proposals  are 
considered  responsive  to  the 
solicitation  and  determined  to 
be  responsible  (capable)  are 
kept  in  the  competitive  ranqe. 

Responsibility  is  generally  pre- 
determined 

-  only  the  best  performing 
vendors  are  given  opportunity 
to  provide  bids,  quotes,  or 
proposals. 

These  vendors  are  then 
evaluated,  similar  to  industry 
according  to: 

These  vendors  are  then 
evaluated,  similar  to 
government  according  to: 

1.  Technical  Capability 

1.  Technical  Capability 

2.  Past  Performance 

2.  Past  Performance 

3.  Cost/Price 

3.  Cost/Price 

Table  14  .  Summary  of  Government  v.  Industry  PPI  Evaluation  Methodology 
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Although  the  entire  acquisition  process  is  similar  between  government  and 
industry,  the  methodology  by  which  each  group  selects  who  will  compete  is  decidedly 
different.  As  shown  in  Tables  13  and  14,  one  of  the  key  differences  between 
government  and  industry  lies  in  the  determination  of  responsibility. 

INDUSTRY.  As  explained  earlier,  industry  has  effectively  eliminated  this  step  by 
choosing  to  compete  only  among  those  vendors  who  they  know  to  be  responsible 
(capable)  of  performance  if  given  the  opportunity.  Industry  also  knows,  based  on  past 
experience  with  a  particular  vendor,  that  the  vendor’s  incoming  bid,  quote,  or  proposal 
will  be  responsive  (timely  and  with  a  solid  understanding  of  the  requirement). 

It’s  vital  to  note,  however,  that  Industry  firms  who  receive  government  contracts 
are  subject  to  flow-down  clauses  in  the  administration  of  subcontractors.  For  example, 
the  PCO  may  require  industry  to  allocate  a  portion  or  percentage  of  the  contract  value 
to  Small  Businesses.  Subsequent  to  this  requirement,  the  industry  firm’s  subcontract 
manager  must  seek  out  qualified  SB  vendors,  under  the  scrutiny  of  the  government’s 
Contract  Administration  Office.  Additionally,  the  industry  firm  may  be  required  to 
maintain  an  approved  purchasing  system  for  the  administration  of  contract  performance, 
an  approved  accounting  system,  etc.  Although  SB  requirements  may  alter  the  way 
industry  awards  and  administers  subcontracts  of  government  awards,  the  majority  of 
subcontracting  appears  to  be  performed  at  the  prerogative  of  the  industry  procurement 
official.44  The  fact  remains,  however,  that  the  effect  of  flow-down  clauses  is  a  factor  in 
how  industry  administers  subcontracts;  yet,  the  process  in  full  is  outside  the  scope  of 
this  study. 

GOVERNMENT.  Government,  on  the  other  hand,  must  laboriously  proceed  with 
a  lengthy  evaluation  of  all  bids,  quotes  or  proposals  in  order  make  a  responsive  and 
responsibility  determination.  If  government  receives  a  cumbersome  number  of 
proposals,  all  must  be  evaluated  equally;  however,  the  Procuring  Contracting  Officer 


44  Lockheed  Martin,  Opalsoft,  Northrop  Grumman  Newport  News,  Maersk  Line  Ltd.,  interviews  conducted 
Aug  -  Sep  2004. 
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(PCO)  has  the  option  of  eliminating  those  proposals  which,  in  his/her  determination, 
have  little  chance  of  winning  contract  awards.  This  process  eliminates  vendors  from  the 
competitive  range  and  from  further  consideration  for  contract  award.45  All  disappointed 
offerors  have  the  right  to  a  pre-award  debrief,  and  have  the  right  to  protest  or  file  claims. 

Therefore,  if  government  wishes  to  improve  its  procedures  for  awarding 
contracts,  it  need  look  no  farther  than  the  process  illustrated  in  Figure  2  -  Government 
v.  Industry  Procurement  Action  Lead  Time  Comparison;  the  industry  models  depicted  in 
Table  13  -  Summary  of  Government  v.  Industry  PPI  Collection  Systems;  and  Table  14  - 
Summary  of  Government  v.  Industry  PPI  Evaluation  Methodology. 


45  Federal  Acquisition  Regulation  (FAR)  Part  15.306(c) 
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CHAPTER  VI.  RECOMMENDATIONS  AND 
CONCLUSIONS 


A.  RECOMMENDATIONS 

Based  on  interviews  with  government  and  industry  acquisition  professionals  in 
the  field  and  data  collected  from  those  interviews,  the  researcher  makes  the  following 
recommendations: 

1.  Government  should  continue  building  and  improving  on  the  existing 
government  PPI  collection  system.  Although  the  Government  acquisition  process 
remains  vulnerable  to  vendor  protests  and  claims,  having  a  robust  government-wide 
PPI  database  populated  with  current  and  relevant  vendor  past  performance  history, 
although  burdensome,  is  worth  maintaining.  Rationale:  Technical  capability/quality  is 
normally  the  highest-weighted  factor  in  a  best-value  source-selection  acquisition. 

Vendor  past  performance  typically  also  carries  considerable  weight.  If  the  PCO  elects 
to  eliminate  a  vendor,  or  group  of  vendors,  from  the  competitive  (prior  to  contract 
award),  the  disappointed  vendor(s)  may  elect  to  file  a  protest(s).  If  those  vendors 
eliminated  had  strong  or  comparable  technical/quality  proposals,  yet  all  had  histories  of 
poor  quality  and  delivery,  the  PCO  will  prevail  and  the  contract  requirement  will  continue 
to  move  forward.  If  past  performance  was  not  considered,  it  is  feasible  that  a  vendor 
could  write  a  good  proposal  (perhaps  “parroting”  the  requirement)  and  provide  a  cost  or 
price  which  is  artificially  low,  thereby  winning  the  contract  award,  only  to  be  terminated 
for  default  (T4D)  due  to  substandard  contract  performance.  Retaining  past  performance 
as  a  major  factor  in  source  selection  is  the  only  viable  means  for  preventing  this  type  of 
occurrence. 


FAR  Part  15  has  given  the  Government  PCO  the  authority  to  eliminate  non- 
responsive  and  non-responsible  offerors  from  the  competitive  range  or  from  final 
contract  award.  However  a  robust  PPI  system  must  be  in  place  in  order  to  back  up  the 
PCO  when  challenged.  For  this  reason,  government  must  continue  to  move  forward  in 
its  objective  to  provide  a  reliable  PPI  collection  and  evaluation  system. 
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2.  Industry  should  continue  with  a  scorecard  PPI  system.  Industry  has  the 
capability  of  conducting  responsibility  (capability)  determinations  on  new  vendors, 
eliminating  non-responsible  vendors  from  further  consideration,  and  selecting  a  vendor 
based  on  a  best  value-trade  off  approach;  yet,  industry  simply  does  not  conduct  full- 
scale  best-value  source-selections  often  enough  to  warrant  an  elaborate  PPI  collection 
system.  Industry  does  not  have  a  CICA  (full  and  open  competition)  requirement,  nor 
does  it  have  socioeconomic  goals  to  achieve.  The  researcher  concludes  that  industry 
should  maintain  a  system  of  documenting  vendor  past  performance — yet  more  for  the 
purpose  for  imposing  process  improvements  amongst  its  critical  path  vendors  than  in 
determining  awards.  The  threat  of  senatorial  or  congressional  inquiries  for  arbitrarily 
awarding  contract  requirements  to  “A  Team”  vendors  is  not  great  enough  to  warrant  an 
elaborate  PPI  system. 

3.  Competition  in  Contracting  Act  (CICA)  of  1984  is  in  need  of  revision. 

Industry  has  a  decided  advantage  in  Procurement  Action  Lead  Time  (PALT)  when 
compared  to  government.  Although  PALT  is  a  somewhat  dated  measure  of 
procurement  efficiency,  it  does  bring  to  light  an  obvious  mismatch  between  the  public 
and  private  sectors.  If  government  is  willing  to  accept  a  210  PALT  when  industry  is 
awarding  the  same  contract  requirement  in  75  days,  then  change  is  not  needed. 
However,  if  government  is  serious  about  positive  change,  CICA  and  government  socio¬ 
economic  policies  must  be  revisited.  Major  reasons  for  industry’s  decided  advantage 
are  the  following  issues:  1 )  the  right  to  provide  a  quote,  bid,  or  proposal,  2) 
government’s  mandate  to  use  full  and  open  competition  whenever  possible,  and  3) 
socioeconomic  policy.  In  industry,  as  discussed  earlier,  vendors  are  privileged  to  be 
invited  to  submit  a  quote,  bid,  or  proposal.  This  privilege  is  only  extended  to  industry’s 
“A  Team”  vendors. 

Areas  for  Further  Research: 

-  A  study  of  the  impact  of  “shifting”  the  burden  of  attaining  small  business  (SB) 
set-aside  socioeconomic  policy  goals  from  government  to  the  prime 
contractors  —  and  perhaps  raising  SB  goals  to  50%  of  the  contract  value. 
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-  A  study  of  the  impact  of  raising  the  threshold  for  full  and  open  competition 
from  $2,500  to  $50,000. 

In  summary,  the  notion  is  to  study  the  feasibility  of  shifting  from  a  “right  to  bid”  to 
a  “privilege  to  bid”  model  and  to  shift  the  requirement  of  government  to  maintain 
socioeconomic  policy  to  that  of  industry 

Rational.  Government  PCO’s  need  the  tools  to  allow  them  to  move  quickly  but 
responsibly  in  order  to  keep  up  with  rapidly  changing  requirements  in  a  rapidly  changing 
world.  Program  Managers  and  end-users  need  to  be  able  to  rely  on  a  highly 
responsive,  yet  highly  ethical  procurement  system. 

Discussion.  Again,  the  question  raised  is  that  of  efficiency.  Is  210  days  to  award 
a  contract  requirement  good  enough  when  75  days  is  possible?  Will  the  process  to 
safeguard  this  radical  acquisition  process  in  order  to  achieve  efficiency  be  too 
cumbersome?  How  would  government  counter  the  risk  of  abuse?  Is  a  new  culture,  a 
new  group  of  people,  a  new  group  of  business  skills  required  to  conduct  these 
tradeoffs?  What  is  the  limit  of  acceptability  of  risk  we  in  government  are  willing  to 
assume?  Would  the  SB  world  revolt  if  it  lost  its  set-aside  threshold  of  $2,500  - 
$100,000  in  favor  of  more  aggressive  SB  subcontracting  goals  placed  on  prime 
contractors?  Again,  these  areas  require  continued  research. 

4.  Continued  compliance  with  established  policies  in  reporting  past- 
performance  history  is  required.  As  discussed  briefly  above,  one  of  the  main 
difficulties  of  using  past  performance  information  from  a  government  database  is  the 
lack  of  data  about  individual  vendors.  This  problem  appears  to  be  most  pronounced  in 
field  activities,  and  to  a  much  lesser  degree,  in  larger  systems  commands  where  the 
number  of  prime  contractors  are  fewer  in  number. 


Discussion.  Searching  the  PPIRS  database  for  history  on  a  particular  vendor 
only  to  find  information  which  is  sketchy,  missing  and  perhaps  inflated  drives  the 
Technical  Evaluation  Board  or  the  Source  Selection  Evaluation  Board  to  seek  PPI  from 
alternate  sources  including  telephone  interviews  and  written  questionnaires.  This 
process  is  laborious  and  slows  the  evaluation  process;  therefore,  each  government 
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contracting  activity  must  remain  diligent  in  maintaining  the  government-wide  PPI 
database.  Although  PPIRS  is  not  a  prefect  system,  government  procurement  personnel 
must  work  with  the  tools  given  and  ensure  information  forwarded  is  timely  and  accurate. 


B.  CONCLUSIONS 

Throughout  this  field  study,  the  researcher  has  noted  observations  in  both 
government  and  industry  regarding  how  each  collects  and  evaluates  PPI  for  the 
purpose  of  making  a  valid  assessment  of  risk.  Following  is  a  summary  of  these 
conclusions: 


•  Industry  is  probably  where  it  needs  to  be  regarding  collection  and  use  of  vendor 
past- perform  a  nee  history.  A  passive  “somewhat  hand-off”  PPI  collection  system 
centered  around  a  closed-loop  purchasing-shipping-receiving-QC  system  is  probably 
sufficient  given  industry’s  volume  of  procurement  activity  and  current  lack  of 
competition  or  socioeconomic  requirements.  Also,  the  level  of  PPI  infrastructure 
should  be  tailored  to  the  relative  size  of  the  industry  firm. 

•  Government’s  PPI  collection  system  is  much  larger,  more  robust,  and  exceedingly 
more  man-power  intensive  than  industry’s.  This  “robust”  system  is  required  if 
government  is  to  successfully  award  to  other  than  the  lowest  priced  technically 
acceptable  offeror  and  in  turn  successfully  defend  such  actions  when  challenged  by 
vendor  protests.  Although  government  is  always  on  the  path  to  re-invent  itself,  it  is 
not  likely  that  it  will  come  much  closer  to  adopting  industry  PPI  procedures  than 
where  it  currently  stands. 

•  Opportunity  for  improvements:  Acquisition  cycle-time  or  PALT.  Being  afforded  the 
opportunity  to  converse  directly  with  industry  acquisition  professionals  at  their  own 
respective  sites  yielded  one  major  lasting  impression  on  the  researcher.  That  lasting 
impression  centers  around  the  speed  and  efficiency  in  which  industry  operates  in  the 
acquisition-planning,  proposal-evaluation,  source-selection,  and  contract-award 
processes.  It  is  obvious  that  industry  has  the  competitive  advantage  in  acquisition 
cycle-time.  The  only  way  to  address  this  shortcoming  of  the  governmental  process 
is  to  address  government-mandated  competition  requirements  and  socioeconomic 
goals. 

As  the  world  situation  continues  to  change,  the  way  we  fight  wars  must 
subsequently  change.  Key  to  that  effort  is  acquisition.  Government  must  take  bold 
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steps  to  evolve  the  acquisition  process  into  an  agile,  efficient,  streamlined  process  if  it  is 
to  remain  responsive  to  both  the  war  fighter  and  the  taxpayer. 


j 


X^^RAtSTANTIA  PER  SCffNrZu^/ 
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Intentionally  Left  Blank 


Appendix  1:  FIELD  INTERVIEW/SURVEY— Past 
Performance  Information  (PPI) 


Activity: _ Date: 

POC:  _ 


This  study  will  compare  DoD  and  industry  past  performance  measures  with 
the  goal  of  improving  the  DoD  system,  to  better  reflect  sound  business  practices. 

Sponsor:  Program  Executive  Officer — Ships 

Researcher:  CDR  Roger  Lord,  SC,  USN,  CPCM,  Graduate  School  of  Business  & 
Public  Policy,  Naval  Postgraduate  School,  Monterey,  CA 

SECTION  1— GENERAL 

1.  In  your  words,  describe  your  organization’s  mission: 

Notes: 


2.  In  your  words,  describe  where  your  contracting  and  acquisition  group  fits  in 
your  organization: 


Notes: 
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Primarily  Large  Contracts? _ Volume/year _ 

Primarily  Small  Contracts? _ Volume/year _ 

SECTION  2— PAST  PERFORMANCE  INFORMATION— COLLECTION 

3.  Is  contractor/firm  PPI  collected  from  each  contract?  What  kinds  of  PPI  data  do 
you  collect  prior  to  source  selection  (from  an  active  contract-in-progress)? 

Notes: 


a.  Business  relations: 

b.  Quality: 

c.  Schedule: 

d.  Cost  control: 

Notes: 


e.  Frequency  of  evaluations:  _ (e.g.  annually,  end  of  contract,  significant 

event/milestone)? 

f.  Dollar  thresholds:  _ (e.g.  all  procurements  over  $25,000)? 


Notes: 


4.  PPI  data  management: 


a.  How  is  PPI  data  collected/accumulated? _ 

b.  How  is  PPI  data  managed  and  available  (distribution): 
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5.  RFP  preparation  -  PPI  data  collection: 


a.  Is  PPI  data  requested  in  each  RFP? _ 

-  Relevancy  (similar  in  scope):  _ 

-  Currency  (<3  yrs):  _ 

-  Quality  (good  performance):  _ 

-  Experience  (#  of  years  in  the  ind): _ 

-  Other  factors:  _ 

b.  Do  you  stipulate  intention  to  conduct  research  of  named  references? 

-  Telephone  interviews:  _ 

-  Questionnaires:  _ 

c.  Do  you  stipulate  intention  to  use  other  than  named  references? 

-  End  users:  _ 

-  Other  customers:  _ 

-  Passive  PPI  (i.e.  QDRs):  _ 

-  Site  visits:  _ 

-  Pre-award  surveys:  _ 

Notes: 


6.  Discuss  your  experience  with  CPARS  as  a  source  of  PPI: 


Notes: 


a.  Difficulties  encountered: 


b.  Methods  for  improvement: 
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7.  If  not  CPARS,  what  sources  (or  industry  model)  do  you  use  to  gather  and 
maintain  PPI  on  a  contractor  (or  firm)? 

Notes: 


a.  Is  your  system  streamlined  (user  friendly)? _ 

b.  Automated? _ 

c.  Web-based? _ 

d.  Kept  current? _ 

e.  Recorded  for  each  contract  at  least  annually,  interim,  and  final? _ 

f.  Contractor  given  opportunity  to  comment  on  PPI  appraisals? _ 

g.  Performance  assessments  discussed  during  initial  post  award  conference? 


Notes: 


SECTION  3:  PAST  PERFORMANCE  INFORMATION— EVALUATION  (INITIAL 
PROCUREMENT) 

8.  Who  typically  comprises  your  acquisition  team  for  an  initial  procurement  of 
estimated  amount:  (e.g.  $1M  or  more,  $100K  for  smaller  firms)? 


a.  Procuring  Contracting  Officer: _ 

b.  Legal  counsel:  _ 

c.  Technical  Evaluation  Board:  _ 

d.  Past  Performance  Board:  _ 

-  Separate  (Contracting  AND  Program  Office  reps) 

-  Notes: _ 

e.  Cost/Price  Analyst:  _ 

f.  End  User: 
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g.  Engineering  Rep:  _ 

h.  Logistics  Rep:  _ 

i.  Comptroller/Budget:  _ 

j.  Small  Business  Rep:  _ 

k.  Quality  Assurance  Rep:  _ 

9.  Is  PPI  a  factor  (or  sub-factor)  in  your  initial  procurements? _ 

The  objective  of  evaluating  PPI  is  to  determine  the  level  of  risk  involved  in 
acceptance  of  each  offeror’s  proposal. 

a.  Resident  in  your  Request  for  Proposal  (RFP)?  Yes _ No _ 

b.  If  yes,  what  is  the  relative  weight  compared  to  other  source  selection  factors? 

□  4.  D3.  U2.  Ell. 

Highest  Weight  Lowest  Weight 

c.  In  a  Best  Value-tradeoff  approach  procurement,  how  do  you  typically  rank 
past  performance  relative  to  other  factors? 

Example:  1)  Technical  Capability  (Most  important) 

2)  Past  Performance 

3)  Other  factors 

4)  Cost/Price  (Least  important) 

Notes: 
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10.  Has  Past  Performance  of  a  contractor  (or  firm)  ever  been  the  deciding  factor 
for  elimination  from  the  competitive  range  or  non-selection  for  contract  award? 


a.  Has  legal  recourse/challenges  been  experienced  IRT  use  of  PPI  as  a  basis 
for  non-selection? 

Notes: 


b.  If  so,  did  your  organization  provide  satisfactory  justification  for  its  decision  not 
to  select  a  contractor  (or  firm)? 

Notes: 


11.  Does  your  organization  have  an  established  list  of  favorite  “go-to” 
contractors  (or  firms)?  _ 

a.  How  does  a  contractor  (or  firm)  get  on  the  list  (i.e.  bound  by  law)? 

b.  How  does  a  contractor  (or  firm)  stay  on  the  list? 

c.  How  does  a  contractor  (or  firm)  get  removed  from  the  list  (i.e.  suspend  or 
debar,  defacto  debarment)? 

d.  Can  a  contractor  (or  firm)  continue  making  offers  (bids/proposals)  after  being 
removed  from  the  go-to  list  OR  are  they  out  forever? 

Notes: 


fRAtSTANHA  PtR  i 
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e.  Does  your  activity  or  firm  use  PPI  data  as: 

i.  Comparative  (relevant,  current,  quality) 

OR  only  as 

ii.  Pass/Fail  (i.e.  on  Suspension,  debarment) 

Notes: 


12.  RFP  evaluation  factors — PPI  data  evaluation: 

a.  Is  PPI  data  requested  in  RFP  evaluated  according  to  the  following: 

-  Relevancy  (similar  in  scope):  _ 

-  Currency  (<3  yrs):  _ 

-  Quality  (good  performance):  _ 

-  Experience  (#  of  years  in  the  ind): _ 

-  Other  factors:  _ 

b.  Methods  used  to  conduct  research  of  named  references? 

-  Telephone  interviews:  _ 

-  Questionnaires:  _ 

c.  Actual  use  of  other  than  named  references? 

-  End  users:  _ 

-  Other  customers:  _ 

-  Passive  PPI  (i.e.  QDRs):  _ 

-  Site  visits:  _ 

-  Pre-award  surveys:  _ 

Notes: 
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Appendix  2:  PAST  PERFORMANCE  DOLLAR  THRESHOLDS 


Past  performance  must  be  included  as  an  evaluation  factor  in  government 
procurements  for  specific  procurements  with  specific  anticipated  contract  values,  as 
follows: 

-  Systems 


-  Services 

-  Operation  Support 

-  Fuels 

-  Healthcare 

-  Information  Technology 

-  Construction 

-  Architect-Engineering 

-  Science  and  Technology 


>  $5,000,000 

(including  new 
development  and  major 
modifications) 

>  $1,000,000 

>  $5,000,000 

>  $100,000 
>  $100,000 
>$  1,000,000 

>  $500,000 

>  $25,000 
As  required 
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APPENDIX  3:  PUBLIC  SECTOR  (DoD)  ACTIVITY  #1: 
Naval  Sea  System  Command  (SEA-02) 

Section  I— GENERAL 

1 .  Organization  Mission.  The  primary  mission  of  NAVSEA-02 
(Contracts  Directorate)  is  direct  contract  support  for  the  procurement  of  U.S. 
Navy  ships,  submarines,  and  their  associated  systems. 

2.  Contract  organization  fit  within  the  organization.  Within  Naval  Sea 
Systems  Command  (NAVSEA),  the  contracting  activity  is  coded  as  a  directorate 
and  fits  as  a  1st  tier  department  on  equal  footing  with  other  directorates  (e.g. 
Comptroller,  Engineering,  Logistics/Maintenance  etc).  The  Contracting 
Directorate  exercises  a  high  degree  of  autonomy.  Each  Procuring  Contracting 
Officer  (PCO)  and  Contract  Negotiator  is  “detailed  out”  to  support  major 
programs  within  NAVSEA  (e.g.  DD(X),  CVN-21,  LCS,  CG(X)  etc.) 

Section  II— PPI  COLLECTION 

3.  Is  PPI  collected  on  each  contractor/firm  prior  to  each  contract?  Yes,  on 
all  requirements  requiring  formal  contract  procedures  (i.e.  non-commercial 
requirements  anticipated  at  greater  than  $100,000,  or  commercial  items  with  an 
anticipated  dollar  value  >$5M),  the  following  PPI  data  is  evaluated  using  CPARS, 
and  other  sources: 


-  Business  Relations: 

Yes 

-  Ouality: 

Yes 

-  Schedule: 

Yes 

-  Cost  Control: 

Yes 
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4.  PPI  data  management.  PPI  data  is  collected  from  contract  actions 


exceeding  the  thresholds  listed  in  Appendix  2,  Past  Performance  Information 
Thresholds.  Feedback  to  the  contractor  via  CPARS  is  the  responsibility  of  the 
Program  Manager. 

-  How  is  PPI  data  collected:  CPARS  is  the  primary  mechanism  for  collecting 
PPI  data. 


-  How  is  PPI  data  managed  and  available:  Automated  PPIRS  database  via 
password  security. 


5.  Request  for  Proposal  (RFP)  preparation. 

a.  Is  PPI  data  requested  in  each  RFP?  Yes,  the  RFP  is  more  specific 
and  spells  out  the  factors  and  range  of  data  that  will  be  considered. 


-  Relevancy  (similar  in  scope):  Yes 

-  Currency  (<3  yrs):  Yes 

-  Quality  (good  performance):  Yes 

-  Experience  (#  of  years  in  the  ind):  Yes 

-  Other  factors:  Yes 


b.  Do  you  stipulate  intention  to  conduct  research  of  named  references? 

Yes 


-  Telephone  interviews:  Yes 

-  Questionnaires:  Yes 

b.  Do  you  stipulate  intention  to  use  other  than  named  references? 

-  End  users:  Yes 

-  Other  customers:  Yes 

-  Passive  PPI  (i.e.  QDRs):  Yes 

-  Site  visits:  N/A 

-  Pre-award  surveys:  No 
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PCO  DD(X)  comments:  CPARS  quality.  If  a  potential  contractor’s  CPARS  scores 
do  not  match  scores  of  other  CPARS  raters,  telephone  calls  are  made  by  the 
reviewer  to  seek  out  the  problem  and  determine  whether  this  discrepancy  is  a 
serious  problem  before  proceeding. 

6.  Discuss  your  experience  with  CPARS  as  a  source  of  PPI: 

a.  Comments:  CPARS  data  is  only  good  if  it  reflects  actual  performance. 
PM’s  are  sometimes  hesitant  to  give  the  contractor  lower  ratings  due  to 
program  risks  (potential  budget  cuts).  A  budget  cut  may  be  brought  on 
by  perceived  risk  in  the  PM’s  program  difficulties  encountered. 

b.  Methods  for  improvement:  PCO  DD(X)  comments.  Checking  the  history 
of  Award  Fees  for  a  potential  contractor  would  shed  considerable  light 
on  a  particular  contractor’s  actual  performance. 

7.  If  not  CPARS,  what  sources  (or  industry  model)  do  you  use  to  gather  and 
maintain  PPI  on  a  contractor  or  firm?  N/A — CPARS  used 

SECTION  3:  PAST  PERFORMANCE  INFORMATION— EVALUATION  (INITIAL 
PROCUREMENT) 

8.  Who  typically  comprises  your  acquisition  team  for  an  initial  procurement  of 
estimated  amount  (e.q.  $1M  or  more)? 


a.  Procuring  Contracting  Officer:  X _ 

b.  Legal  counsel:  X _ 

c.  Technical  Evaluation  Board:  X _ 

d.  Past  Performance  Board:  X _ 

Separate  (Contracting  AND  Program  Office  reps) 

e.  Cost/Price  Analyst:  X _ 

f.  End  User/Engineering  Rep:  X _ 

g.  Logistics  Rep:  X _ 

h.  Comptroller/Budget:  X _ 

i.  Small  Business  Rep:  X _ 

j.  Quality  Assurance  Rep:  _ 


9.  Is  PPI  a  factor  (or  sub-factor)  in  your  initial  procurements?  YES 
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a.  Resident  in  your  Request  for  Proposal  (RFP)?  Yes  X  No _ 

b.  If  yes,  what  is  the  relative  weight  compared  to  other  source  selection  factors? 


□  4. 

□  3. 

□ 

c\i 

X 

Highest  Weight 

Lowest  Weight 

c.  In  a  Best  Value-tradeoff  approach,  how  do  you  typically  rank  past 
performance  relative  to  other  factors? 

1 )  Technical  Capability  (Most  important) _ 

2)  Other  factors 

3)  Past  Performance 

4)  Cost/Price  (Least  important) 

PCO  DD(X)  comments:  See  Sections  L  and  M  of  sample  RFP 

10.  Has  Past  Performance  of  a  contractor  or  firm  ever  been  the  deciding  factor 
for  elimination  from  the  competitive  range  or  non-selection  for  contract  award?  NO 

a.  Have  legal  recourse/challenges  been  experienced  IRT  use  of  PPI  as  a 
basis  for  non-selection?  N/A 

b.  If  so,  did  your  organization  provide  satisfactory  justification  for  its 
decision  not  to  select  a  contractor  or  firm?  N/A 

1 1 .  Does  your  organization  have  an  established  list  of  favorite  “go-to” 
contractors  or  firms?  There  is  a  list  of  contractors  eligible  to  become  “prime 
contractors”  in  the  shipbuilding  industry;  however,  the  list  of  small.  As  little  as 
six  firms  are  “capable”  of  designing,  integration,  and  building  U.S.  Navy  ships 
today. 


If  firm  is  suspended  or  debarred,  no  bids  or  proposals  will  be  entertained. 
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a.  How  does  a  contractor  or  firm  get  on  the  list?  Become  capable. 


b.  How  does  a  contractor  or  firm  stay  on  the  list?  Stay  capable. 

c.  How  does  a  contractor  or  firm  get  removed  from  the  list  ?  If 

suspended  or  debarred. 

d.  Can  a  contractor  or  firm  continue  making  offers  (bids/proposals)  after 
being  removed  from  the  go-to  list  OR  are  they  out  forever?  If  suspended  or 
debarred,  the  contractor  is  ineligible  to  receive  a  contract  award. 

Notes: 

Can  continue  to  forward  bid,  quotes,  or  proposals  unless  suspended  or 
debarred. 

Does  your  firm  use  PPI  data  as: 

ii.  Comparative  (relevant,  current,  quality) 

OR  only  as 

iii.  Pass/Fail  (i.e.  on  Suspension,  debarment) 

PCO/SSA  comments.  In  formal  contract  procedures  (greater  than  $100,000, 
or  commercial  procurements  currently  greater  than  $5,000,000)  a  two-step 
process  occurres.  First,  the  firm  must  not  be  on  the  Suspended  or  Debarred  list; 
second,  the  firm’s  relevant  performance,  currency,  and  quality  is  reviewed. 

12.  RFP  evaluation  factors — PPI  data  evaluation: 
a.  PPI  data  requested  in  RFP?  YES 

-  Relevancy  (similar  in  scope):  YES 

-  Currency  (<3  yrs):  YES 

-  Quality  (good  performance):  YES 

-  Experience  (#  of  years  in  the  ind):  YES 

-  Other  factors: 
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b.  Stipulate  intention  to  conduct  research  of  named  references? 


-  Telephone  interviews:  YES 

-  Questionnaires:  YES 

c.  Stipulate  intention  to  use  other  than  named  references? 

-  End  users:  YES 

-  Other  customers:  YES 

-  Passive  PPI  (i.e.  QDRs):  YES 

-  Site  visits:  N/A 

-  Pre-award  surveys:  YES 


PCO/DD(X)  comments.  Stipulating  in  Section  L  the  intention  to  seek  out 
past  performance  history  outside  the  normal  channels  of  CPARS,  RYG,  or  any 
other  government  database  is  prudent  and  gives  ALL  offerors  “fair  warning.” 
This  stipulation  has  benefits  in  the  face  of  a  protest  where  past  performance  is 
used  as  the  basis  for  elimination  from  the  competitive  range  or  caused  the  firm 
not  to  receive  the  contract  award. 
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APPENDIX  4:  PUBLIC  SECTOR  (DoD)  ACTIVITY  #2:  Naval 
Air  Systems  Command  (AIR-2.0  &  4.1  OE) 


Section  I— GENERAL 

1 .  Organization  Mission.  The  primary  mission  of  NAVAIR-2.0 
(Contracts  Directorate)  is  direct  contract  support  for  the  procurement  of  U.S. 
Navy  Tactical  Aircraft,  Non  Tactical  Aircraft,  and  Aircraft  Weapons  Systems. 
NAVAIR  4.10E  comprises  a  standing  Source  Selection  Evaluation  Board  (SSEB) 
for  NAVAIR. 

2.  Contract  organization  fit  within  the  organization.  Within  the  Naval 
Air  Systems  Command  (NAVAIR),  the  contracting  activity  is  coded  as  a 
directorate  and  fits  as  a  1st  tier  department  on  equal  footing  with  other 
directorates  (e.g.  Comptroller,  Engineering,  Logistics/Maintenance  etc).  The 
Contracting  Directorate  exercises  a  high  degree  of  autonomy.  Each  Procuring 
Contracting  Officer  (PCO)  and  Contract  Negotiator  is  detailed-out  to  support 
major  programs  within  NAVAIR  (e.g.  F-35  (JSF),  FA-18E/F,  MMA  etc.).  NAVAIR 
has  taken  the  lead  in  the  source-selection  process.  The  majority  of  comments  in 
Section  II  below  are  derived  from  a  field  interview  conducted  with  a  Chairman  of 
many  of  the  recent  Source  Selection  Evaluation  Boards  (SSEB),  located  in  AIR 
4.10E. 

Section  II— PPI  COLLECTION 

3.  Is  PPI  collected  on  each  contractor/firm  prior  to  each  contract? 

Yes,  on  all  requirements  requiring  formal  contract  procedures.  The  following  PPI 


data  is  collected  using  CPARS. 

-  Business  Relations:  Yes 

-  Quality:  Yes 

-  Schedule:  Yes 

-  Cost  Control:  Yes 


4.  PPI  data  management. 

-  How  is  PPI  data  collected:  PPI  data  is  collected  from  all  eligible 
contract  actions.  Data  is  credited  to  the  contractor  by  Commercial  and 
Government  Entity  (CAGE)  code,  thereby  giving  credit  to  the  branch,  division  (or 
cost  center)  which  performed  the  majority  of  work.  Feedback  to  the  contractor 
via  CPARs  is  the  responsibility  of  the  Program  Manager.  The  contractor  has  30 
days  to  respond.  Disputes  are  handled  at  a  level  above  the  PM. 

-  How  is  PPI  data  managed  and  available:  Automated  PPIRS/CPARS 
database  visible  only  to  government  officials  with  a  need  to  know. 

6.  Reguest  for  Proposal  (RFP)  preparation. 
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a.  Is  PPI  data  requested  in  each  RFP?  Yes,  the  RFP  is  more  specific 
and  spells  out  the  factors  and  range  of  data  that  will  be  considered. 

-  Relevancy  (similar  in  scope):  Yes 

-  Currency  (<3  yrs):  Yes  (if  contract  is  still  open) 

-  Quality  (good  performance):  Yes 

-  Experience  (#  of  years  in  the  ind):  Yes  (treated  as  a  separate  eval 

factor) 

-  Other  factors: 

b.  Do  you  stipulate  intention  to  conduct  research  of  named  references? 

Yes 

-  Telephone  interviews:  No 

-  Questionnaires:  Yes  (has  prime  contractor  send 

out  questionnaires,  coded  back  to  NAVAIR) 

c.  Do  you  stipulate  intention  to  use  other  than  named  references? 

-  End  users:  Yes  (other  PCOs  or  ACOs) 

-  Other  customers:  Yes 

-  Passive  PPI  (i.e.  QDRs):  No 

-  Site  visits:  N/A 

-  Pre-award  surveys:  Yes  (if  vendor  unknown  &  it 

makes  sense) 

SSEB  Chairman  comments:  CPARS  quality.  If  a  potential  contractor’s  CPARS 
scores  do  not  match  scores  of  other  CPARS  raters,  telephone  calls  are  made  to 
the  reviewer  to  seek  out  the  problem  and  a  determination  is  made  as  to  whether  it 
is  a  serious  problem  before  proceeding. 

6.  Discuss  your  experience  with  CPARS  as  a  source  of  PPI: 

a.  Difficulties  encountered.  SSEB  Chairman  comments:  1)  CPARS 
data  is  only  good  if  it  reflects  actual  performance.  2)  PM’s  are  sometimes  hesitant 
to  give  the  contractor  lower  ratings  due  to  potential  budget  cuts.  A  budget  cut 
may  be  brought  on  by  perceived  risk  in  the  PM’s  program  if  contractor  difficulties 
are  encountered.  3)  Quality  of  CPARS  data  is  not  always  consistent  (e.g.  grades 
not  matching  write-ups.  CPARS  data  is  usually  better  (more  consistent)  from 
ACAT  I  or  II  program  vendors  than  from  field  contracting  activity  vendors. 

b.  Methods  for  improvement:  SSEB  Chairman  comments:  1)  Group 
training  on  source  selection  process  including  reading/interpreting  CPARS  data. 
2)  Recommend  PM  prepare  CPARS  response,  PEO  signs  off  on  each  CPARS 
input. 


7.  If  not  CPARS,  what  sources  (or  industry  model)  do  you  use  to  gather 
and  maintain  PPI  on  a  contractor  or  firm?  N/A — CPARS  used 
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SECTION  3:  PAST  PERFORMANCE  INFORMATION— EVALUATION  (INITIAL 
PROCUREMENT) 


8.  Who  typically  comprises  your  acquisition  team  for  an  initial  procurement  of 
estimated  amount  (e.q.  $1M  or  more)? 


a.  Procuring  Contracting  Officer: 

b.  Legal  counsel: 

c.  Technical  Evaluation  Board: 

d.  Past  Performance  Board: 


_X_ 

X 

_ X _ (SSEB) 

_X _ (SSEB) 


Separate  (Contracting  AND  Program  Office  reps) 

e.  Cost/Price  Analyst:  X _ (SSEB) 

f.  End  User/Engineering  Rep:  _ X _ (interprets  requirements, 

interprets  laws  of  physics,  aerodynamics,  etc) 

g.  Logistics  Rep:  X _ 

h.  Comptroller/Budget:  _ 

i.  Small  Business  Rep:  X _ 

j.  Quality  Assurance  Rep:  _ (no,  measured  in  past 

performance) 

9.  Is  PPI  a  factor  (or  sub-factor)  in  your  initial  procurements?  YES 
a.  Resident  in  your  Request  for  Proposal  (RFP)?  Yes  X  No _ 


c. 


b.  If  yes,  what  is  the  relative  weight  compared  to  other  source  selection  factors? 

□  4.  X  3. - X  2.  □  1 . 

Highest  Weight  Lowest  Weight 

In  a  Best  Value-tradeoff  approach,  how  do  you  typically  rank  past 

performance  relative  to  other  factors? 


1)  Technical  Capability  (Most  important) 

2)  Past  Performance  equals  Cost/price 

3)  Other  factors  (Experience)  (Least  important) 

10.  Has  Past  Performance  of  a  contractor  or  firm  ever  been  the  deciding  factor 
for  elimination  from  the  competitive  range  or  non-selection  for  contract  award?  Yes 

(but  rarely) 

a.  Have  legal  recourse  /  challenges  been  experienced  IRT  use  of  PPI  as  a 
basis  for  non-selection?  No 

b. lf  so,  did  your  organization  provide  satisfactory  justification  for  its 
decision  not  to  select  a  contractor  or  firm?  N/A 


1 1 .  Does  your  organization  have  an  established  list  of  favorite  “go-to” 
contractors  or  firms?  There  is  a  list  of  contractors  eligible  to  become  “prime 
contractors”  in  the  aircraft  building  industry.  The  list  of  contractors  are 
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“capabilities  driven”;  therefore,  the  list  is  small.  If  a  firm  is  suspended  or 
debarred,  no  bids  or  proposals  will  be  entertained. 

a.  How  does  a  contractor  or  firm  get  on  the  list?  Become  capable. 

b.  How  does  a  contractor  or  firm  stay  on  the  list?  Stay  capable. 

c.  How  does  a  contractor  or  firm  get  removed  from  the  list  ?  If 

suspended  or  debarred. 

d.  Can  a  contractor  or  firm  continue  making  offers  (bids/proposals)  after 
being  removed  from  the  go-to  list  OR  are  they  out  forever?  If  suspended  or 
debarred,  the  contractor  is  ineligible  to  receive  a  contract  award. 

k.  Does  your  firm  use  PPI  data  as: 

i.  Comparative  (relevant,  current,  guality)  Yes 

a.  OR  only  as 

ii.  Pass/Fail  (i.e.  on  Suspension,  debarment)  Yes 
12.  RFP  evaluation  factors — PPI  data  evaluation: 


a.  PPI  data  reguested  in  RFP? 

YES 

-  Relevancy  (similar  in  scope): 

YES 

-  Currency  (<3  yrs): 

YES 

-  Quality  (good  performance): 

YES 

-  Experience  (#  of  years  in  the  ind):  YES 

-  Other  factors: 

Stipulate  intention  to  conduct  research  of  named  references? 

-  Telephone  interviews: 

No 

-  Questionnaires: 

YES 

Stipulate  intention  to  use  other  than  named  references? 

-  End  users: 

YES 

-  Other  customers: 

YES 

-  Passive  PPI  (i.e.  QDRs): 

YES 

-  Site  visits: 

-  Pre-award  surveys: 

YES 
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APPENDIX  5:  PUBLIC  SECTOR  (DoD)  ACTIVITY  #3:  Fleet 
and  Industrial  Supply  Center,  Norfolk  Naval  Shipyard 
Annex 


Section  I— GENERAL 

1.  Organization  Mission.  Direct  contract  support  for  the  overhaul, 
conversion,  and  repair  of  U.S.  Navy  ships  and  submarines.  Types  of  ships 
include  aircraft  carriers  (CVN  class),  amphibious  assault  ships  (LHA/LHD  class), 
submarines  (SSN  class),  and  a  new  initiative,  the  conversion  of  Trident  Class 
submarines  from  ballistic  missile  capable  (SSBN  class)  to  guided  missile  capable 
(SSGN  class). 

2.  Contract  organization  fit  within  the  organization.  Within  the  Norfolk 
Naval  Shipyard  (NNSY)  organization,  the  Contracting  Division  fits  in  the  2nd  tier  as 
a  division  under  Supply  Department.  The  Supply  Department  is  on  equal  footing 
with  other  shipyard  departments  (e.g.  Engineering  &  Planning,  Operations  & 
Scheduling,  Cranes,  Business  Office  etc).  The  Contracting  Division,  however, 
exercises  a  high  degree  of  autonomy.  Specifically,  each  Procuring  Contracting 
Officer  (PCO)  chooses  the  type  of  contract  vehicle  (e.g.  Firm  Fixed  Price  (FFP), 
Indefinite  Delivery  Type  Contract  (IDTC),  etc.)  and  decides  whether  the 
requirement  can  be  purchased  under  standard  procedures  or  in  a  more 
streamlined  commercial  procedure.  Depending  on  urgency  and  dollar  threshold, 
the  contract  action  may  require  approval  from  a  Contract  Review  Board  (CRB) 
consisting  of  the  Chief  of  the  Contracting  Office  (CCO),  legal  counsel,  and  level 
above  PCO.  No  outside  influence  above  the  Contract  Divisional  level  is  exerted 
regarding  the  type  of  contract  vehicle.  Approximately  3,000  contract  actions  are 
awarded  annually. 

Section  II— PPI  COLLECTION 

3.  Is  PPI  collected  on  each  contractor/firm  prior  to  each  contract?  Yes,  on 
all  requirements  requiring  formal  contract  procedures  (i.e.  non-commercial 
requirements  anticipated  at  greater  than  $100,000,  or  commercial  items  with  an 
anticipated  dollar  value  >$5M),  the  following  PPI  data  is  evaluated  using  CPARS 
maintained  in  the  PEDREP  database  as  well  as  other  sources: 


-  Business  Relations:  Yes 

-  Quality:  Yes 

-  Schedule:  Yes 

-  Cost  Control:  Yes 


For  contracts  or  purchase  orders  less  than  the  above  thresholds, 
NNSY  Contracts  division  uses  a  passive  PPI  collection  system:  the  U.S.  Navy’s 
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Red,  Yellow,  Green  (RYG)  Program  maintained  in  the  PEDREP  database,  in  order 
to  evaluate  the  following: 

-  Business  Relations:  _ 

-  Quality:  Yes 

-Schedule:  Yes 

-  Cost  Control:  _ 


4.  PPI  data  management.  PPI  data  is  collected  from  contract  actions 
exceeding  the  thresholds  listed  in  Appendix  2  Past  Performance  Information 
Thresholds. 

a.  How  is  PPI  data  collected:  CPARS  and  RYG  Program 

b.  How  is  PPI  data  managed  and  available:  Automated  database  via 
password  security. 


5.  Request  for  Proposal  (RFP)  preparation. 


a.  Is  PPI  data  requested  in  each  RFP?  Yes,  the  RFP  is  more  specific  and 
spells  out  the  factors  and  range  of  data  that  will  be  considered. 

-  Relevancy  (similar  in  scope):  Yes 

-  Currency  (<3  yrs):  Yes 

-  Quality  (good  performance):  Yes 

-  Experience  (#  of  years  in  the  ind):  Yes 

-  Other  factors:  Yes 

Do  you  stipulate  intention  to  conduct  research  of  named  references?  Yes 

-  Telephone  interviews:  Yes 

-  Questionnaires:  Yes 

Do  you  stipulate  intention  to  use  other  than  named  references? 

-  End  users:  Yes 

-  Other  customers:  Yes 

-  Passive  PPI  (i.e.  QDRs):  Yes 

-  Site  visits:  Yes 

-  Pre-award  surveys:  Yes  (if  vendor  is  unknown) 

PCO/SSA  Comments:  The  Norfolk  Naval  Shipyard  contracting  division 
aggressively  seeks  out  the  history  of  a  potential  contractor  or  firm  for  contract 
awards  exceeding  $100,000.  For  contract  awards  less  than  $100,000,  the 
contracting  group  depends  highly  on  delivery  and  quality  history  obtained  in  the 
U.  S.  Navy’s  Red,  Yellow,  Green  program. 


b. 


c. 


6.  Discuss  your  experience  with  CPARS  as  a  source  of  PPI: 

The  use  of  CPARS  as  a  method  of  gathering  past  performance  history  is 
cumbersome  and  lends  to  becoming  more  subjective  than  it  probably  should. 


a.  Difficulties  encountered: 

PCO/SSA  comments:  1).  CPARS  is  too  subjective.  Although  currency  (how  recent 
the  contract  was  performed)  is  apparent,  relevancy  (does  the  work  performed 
correlate  to  requirements  in  the  statement  of  work)  remains  the  judgment  of  the 
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past  performance  source-selection  team.  Once  currency  and  relevancy  is 
established  (presumably  in  a  hierarchy  of  most  relevant  to  least  relevant),  a 
determination  must  be  made  as  to  quality  of  the  supply  or  service.  Quality  is 
typically  an  adjective  grade  backed  up  by  a  narrative.  It  is  the  culmination  of 
adjective  grades  and  narrative  comments  on  quality,  cost  control,  schedule,  and 
business  relations  which  can  become  “hair-splitting”  or  otherwise  highly 
subjective.  2)  CPARS  data  is  “hit  and  miss”  on  contractors  whose  contract 
performance  requires  CPARS  surveillance.  Contracting  activity  must  depend  on 
other  sources  of  PPI  in  order  to  piece  together  a  contractor’s  history. 

c.  Methods  for  improvement: 

PCO/SSA  comments:  The  subjectivity  of  CPARS  data  is  probably  not  fixable; 
however,  a  wider  use  of  CPARS  or  another  automated  form  of  maintaining 
contractor  PPI  should  be  mandated.  Perhaps  past  performance  fields  should  be 
added  to  the  DD  Form  350.  Although  including  PPI  on  the  DD  Form  350  would  not 
solve  the  interim  CPARS  reporting  (for  contracts  with  periods  of  performance 
greater  than  one  annual  cycle),  it  would,  however,  improve  the  consistency  of 
reporting  contractor  performance. 

7.  If  not  CPARS,  what  sources  (or  industry  model)  do  you  use  to  gather  and 
maintain  PPI  on  a  contractor  or  firm?  N/A — CPARS/RYG  used 


SECTION  3:  PAST  PERFORMANCE  INFORMATION— EVALUATION  (INITIAL 
PROCUREMENT) 


8.  Who  typically  comprises  your  acquisition  team  for  an  initial  procurement  of 
estimated  amount  (e.q.  $1M  or  more)? 


a.  Procuring  Contracting  Officer:  X _ 

b.  Legal  counsel:  X 

c.  Technical  Evaluation  Board:  X _ 

d.  Past  Performance  Board:  X _ 

-  Separate  (Contracting  AND  Program  Office  reps) 

e.  Cost/Price  Analyst:  X _ 

f.  End  User:  _X_ 

g.  Engineering  Rep:  X _ 

h.  Logistics  Rep:  _ 

i.  Comptroller/Budget:  _ 

j.  Small  Business  Rep:  X _ 

k.  Quality  Assurance  Rep:  _ 


9.  Is  PPI  a  factor  (or  sub-factor)  in  your  initial  procurements?  YES 


a.  Resident  in  your  Request  for  Proposal  (RFP)?  Yes  X  No 


ACQUISITION  RESEARCH 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY 
NAVAL  POSTGRADUATE  SCHOOL 


-84- 


b.  If  yes,  what  is  the  relative  weight  compared  to  other  source  selection  factors? 

□  4.  X  3.  □  2.  I  1. 

Highest  Weight  Lowest  Weight 

c.  In  a  Best  Value-tradeoff  approach,  how  do  you  typically  rank  past 
performance  relative  to  other  factors? 

1)  Technical  Capability  (Most  important) 

2)  Past  Performance 

3)  Other  factors 

4)  Cost/Price  (Least  important) 

PCO/SSA  comments:  Typically  the  end-user  has  a  product  or  process  in  mind.  If 
the  proposals  can  adequately  address  this  product  or  process,  the  technical 
evaluation  board  (which  usually  consists  of  at  least  one  rep  from  the  end-user’s 
division  or  branch),  will  grade  that  particular  proposal  higher  than  others 
exhibiting  lesser  technical  capabilities.  Elimination  from  the  competitive  range  is 
accomplished  after  evaluation  of  the  most  important  source  selection  factor. 

Past  performance  is  of  somewhat  lesser  importance  since  competitors  in  the 
industry  are  typically  known. 

10.  Has  Past  Performance  of  a  contractor  or  firm  ever  been  the  deciding 
factor  for  elimination  from  the  competitive  range  or  non-selection  for  contract  award? 

NO 

a.  Have  legal  recourse/challenges  been  experienced  IRT  use  of  PPI  as  a  basis 
for  non-selection?  N/A 

b.  If  so,  did  your  organization  provide  satisfactory  justification  for  its  decision  not 
to  select  a  contractor  or  firm?  N/A 


1 1 .  Does  your  organization  have  an  established  list  of  favorite  “go-to” 
contractors  or  firms?  There  is  no  “go-to”  list  for  procurements  greater  than  $2,500. 
All  procurements  greater  than  $2,500  must  be  competed.  Procurements  valued  at 
$2,500  -  $100,000  are  set-aside  for  small  business  firms.  Firms  must  obtain 
official  business  status  by  obtaining  a  Dunn  &  Bradstreet  number.  If  a  bid  or 
proposal  is  received  from  a  new  or  unknown  firm,  the  PCO  may  elect  to  deploy  a 
pre-award  survey  team  to  the  firm’s  site  to  make  a  responsibility  determination. 
This  responsibility  determination  may  require  the  survey  team  to  audit:  1)  the 
firms  financial  condition,  2)  accounting  practices,  3)  manufacturing  capabilities, 

4)  quality  assurance,  or  any  other  element  of  concern.  If  the  firm  is  suspended  or 
debarred,  no  bids  or  proposals  will  be  entertained. 

a.  How  does  a  contractor  or  firm  get  on  the  list?  N/A 

b.  How  does  a  contractor  or  firm  stay  on  the  list?  N/A 
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f.  How  does  a  contractor  or  firm  get  removed  from  the  list?  If  suspended  or 
debarred  N/A 

g.  Can  a  contractor  or  firm  continue  making  offers  (bids/proposals)  after  being 
removed  from  the  go-to  list  OR  are  they  out  forever?  N/A 

h.  Does  your  firm  use  PPI  data  as: 

i.  Comparative  (relevant,  current,  guality) 

OR  only  as 

ii.  Pass/Fail  (i.e.  on  Suspension,  debarment) 

PCO/SSA  comments:  In  formal  contract  procedures  (greater  than  $100,000 
or  commercial  procurements  currently  greater  than  $5,000,000),  a  two-step 
process  occurred.  First,  the  firm  must  not  be  on  the  suspended  or 
debarred  list;  second,  the  firm’s  relevant  performance,  currency,  and 
quality  is  considered. 

12.  RFP  evaluation  factors — PPI  data  evaluation: 


a.  PPI  data  requested  in  RFP?  YES 

-  Relevancy  (similar  in  scope):  YES 

-  Currency  (<3  yrs):  YES 

-  Quality  (good  performance):  YES 

-  Experience  (#  of  years  in  the  ind):  YES 

-  Other  factors:  _ 

b.  Stipulate  intention  to  conduct  research  of  named  references? 

-  Telephone  interviews:  YES 

-  Questionnaires:  YES 

c.  Stipulate  intention  to  use  other  than  named  references? 

-  End  users:  YES 

-  Other  customers:  YES 

-  Passive  PPI  (i.e.  QDRs):  YES 

-  Site  visits:  _ 

-  Pre-award  surveys:  YES 


PCO/SSA  comments.  The  intention  to  seek  out  past  performance  history  outside 
the  normal  channels  of  CPARS,  RYG,  or  any  other  government  database  is 
prudent  and  gives  ALL  offerors  “fair  warning.”  This  stipulation  has  the  obvious 
benefits  in  the  face  of  a  protest  where  past  performance  is  used  as  the  basis  for 
elimination  from  the  competitive  range  or  caused  the  firm  not  to  receive  the 
contract  award. 
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APPENDIX  6:  PUBLIC  SECTOR  (DoD)  ACTIVITY  #4 
Tank  Automotive  Command  (TACOM) 


Section  I— GENERAL 

1.  Organization  Mission.  Located  in  Warren,  Ml,  TACOM  is  responsible 
for  the  planning,  acquisition,  fielding,  and  support  for  the  majority  of  the  land 
combat  vehicles  for  the  U.  S.  Army. 

2.  Contract  organization  fit  within  the  organization.  Within  TACOM,  the 
contracting  organization  fits  in  the  organization  as  a  directorate  and  fits  as  a  1st 
tier  department  on  equal  footing  with  other  directorates  (e.g.  Comptroller, 
Engineering,  Logistics/Maintenance  etc).  The  Contracting  Directorate  exercises  a 
high  degree  of  autonomy.  Each  Procuring  Contracting  Officer  (PCO)  and 
Contract  Negotiator  is  detailed-out  to  support  major  programs  within  TACOM. 

Section  II— PPI  COLLECTION 

3.  Is  PPI  collected  on  each  contractor/firm  prior  to  each  contract?  Yes,  on 


all  requirements 

a.  Business  Relations:  Yes 

b.  Ouality:  Yes 

c.  Schedule:  Yes 

d.  Cost  Control:  Yes 


TACOM  comments:  With  adverse  ratings,  TACOM  notifies  the  vendor  if  it 
plans  on  degrading  a  particular  rating,  and  engages  in  discussions. 

4.  PPI  data  management. 

a.  How  is  PPI  data  collected:  PPIRS/CPARS 

b.  How  is  PPI  data  managed  and  available:  PPIRS/CPARS 


5.  Request  for  Proposal  (RFP)  preparation. 

a.  Is  PPI  data  requested  in  each  RFP?  Yes,  the  RFP  is  more  specific  and 
spells  out  the  factors  and  range  of  data  that  will  be  considered. 

-  Relevancy  (similar  in  scope):  Yes 

-  Currency  (<3  yrs):  Yes  (</=  3  years) 

-  Ouality  (good  performance):  Yes  (strength  of  performance) 

-  Experience  (#  of  years  in  the  ind):  Yes  (in  addition  to  past 

performance. 

-  Other  factors:  No 

b.  Do  you  stipulate  intention  to  conduct  research  of  named  references?  Yes. 
RFPs  make  general  statements  that  government  can  gather  past  performance 
data  from  any  source  available  in  order  to  make  a  responsibility  determination. 
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-  Telephone  interviews:  Yes 

-  Questionnaires:  Yes 

c.  Do  you  stipulate  intention  to  use  other  than  named  references? 

-  End  users:  Yes 

-  Other  customers:  Yes 

-  Passive  PPI  (i.e.  QDRs):  Yes 

-  Site  visits:  N/A 

-  Pre-award  surveys:  Yes 


6.  Discuss  your  experience  with  CPARS  as  a  source  of  PPI:  TACOM  uses 
PPIRS/CPARS 

a.  Difficulties  encountered:  1)  Not  many  contracts  in  PPIRS.  TACOM  must 
hold  discussions/exchanges  with  offerors  in  order  to  make  an  accurate  risk 
assessment.  2)  Grade  inflation.  Program  Manager  increases  the  risk  of  program 
cancellation  if  low  grades  are  given  a  contractor  on  interim  reports.  3)  Small 
company  break-offs.  If  a  company  breaks  off  from  a  larger  company,  it  is  difficult 
to  drag  PPI  with  them. 

b.  Methods  for  improvement: 

7.  If  not  CPARS,  what  sources  (or  industry  model)  do  you  use  to  gather  and 
maintain  PPI  on  a  contractor  or  firm?  N/A 


SECTION  3:  PAST  PERFORMANCE  INFORMATION— EVALUATION  (INITIAL 
PROCUREMENT) 


8.  Who  typically  comprises  your  acquisition  team  for  an  initial  procurement  of 
estimated  amount  (e.q.  $1M  or  more)? 


a.  Procuring  Contracting  Officer: 

b.  Legal  counsel: 

c.  Technical  Evaluation  Board: 

d.  Past  Performance  Board: 

e.  Cost/Price  Analyst: 

f.  End  User/Engineering  Rep: 

g.  Logistics  Rep: 

h.  Comptroller/Budget: 

i.  Small  Business  Rep: 

j.  Quality  Assurance  Rep: 


_X_ 

_x_ 

_X_  (Part  of  SSEB) 

_ X _ (Part  of  SSEB) 

_X_  (Part  of  SSEB) 

_ X _ (more  so  on  larger 

procurements) 

_ X _ (concerned  about  ease  of 

maintenance) 


X 
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9.  Is  PPI  a  factor  (or  sub-factor)  in  your  initial  procurements?  YES 

a.  Resident  in  your  Request  for  Proposal  (RFP)?  Yes  X  No _ 

b.  If  yes,  what  is  the  relative  weight  compared  to  other  source  selection  factors? 

□  4.  □  3.  X  2.  □  1 . 

Technical  Past  Performance  =  Price 

Notional  25%  weighting 

Highest  Weight  Lowest  Weight 

c.  In  a  Best  Value-tradeoff  approach,  how  do  you  typically  rank  past 
performance  relative  to  other  factors? 

1)  Technical  Capability  (most  important) 

2)  Cost/Price  AND  Past  Performance 

Notes:  Past  performance  is  usually  not  the  swing,  or  determining  factor  in  a 
source  selection. 

10.  Has  Past  Performance  of  a  contractor  or  firm  ever  been  the  deciding  factor 
for  elimination  from  the  competitive  range  or  non-selection  for  contract  award?  NO 

a.  Have  legal  recourse/challenges  been  experienced  IRT  use  of  PPI  as  a  basis 
for  non-selection?  NO,  usually  not  the  swing  vote 

a.  If  so,  did  your  organization  provide  satisfactory  justification  for  its  decision  not 
to  select  a  contractor  or  firm?  N/A 

1 1 .  Does  your  organization  have  an  established  list  of  favorite  “go-to” 
contractors  or  firms?  No. 


a.  How  does  a  contractor  or  firm  get  on  the  list?  N/A 

b.  How  does  a  contractor  or  firm  stay  on  the  list?  N/A 

c.  How  does  a  contractor  or  firm  get  removed  from  the  list?  N/A 

d.  Can  a  contractor  or  firm  continue  making  offers  (bids/proposals)  after  being 
removed  from  the  go-to  list  OR  are  they  out  forever?  N/A 

e.  Does  your  firm  use  PPI  data  as: 

Comparative  (relevant,  current,  quality)  YES 
OR  only  as 

Pass/Fail  (i.e.  on  Suspension,  debarment)  YES 

12.  RFP  evaluation  factors — PPI  data  evaluation: 


a.  PPI  data  requested  in  RFP?  No  official  RFP  used,  however,  data  required 
for  hiring  (outsourcing)  a  requirement  is  similar. 


-  Relevancy  (similar  in  scope):  YES 

-  Currency  (<3  yrs):  YES 

-  Quality  (good  performance):  YES 

-  Experience  (#  of  years  in  the  ind):  No 
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-  Other  factors:  No 

b.  Stipulate  intention  to  conduct  research  of  named  references? 

-  Telephone  interviews:  YES  (particularly  if  there  is 

adverse  past  performance  history) 

-  Questionnaires:  YES  (tailored  to  the  procurement) 

c.  Stipulate  intention  to  use  other  than  named  references? 

-  End  users:  YES 

-  Other  customers:  YES 

-  Passive  PPI  (i.e.  QDRs):  YES 

-  Site  visits:  N/A 

-  Pre-award  surveys:  YES  (used  as  a  responsibility 

determination) 

TACOM  comments:  Preparing  report  cards:  1)  Past  performance  data  input  is  an 
enormous  burden  for  the  Army  (thousands  of  requirements  under  contract). 
Recommend  raising  the  dollar  threshold  to  $5M  or  above. 

2)  Simplify  CPARS — too  elaborate  for  the  Army.  Probably  won’t  be  able  to  leave 
contractor  [vendor]  out  of  the  input  [process]. 
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APPENDIX  7:  Private  Sector  (Defense-focused) 
Activity  #1:  Lockheed  Martin  Space  Systems 
Company 


Section  I— GENERAL 

1  Organization  Mission.  At  Lockheed  Martin  Space  Systems  Company 
located  in  Sunnyvale,  CA,  the  product  lines  are:  1)  Space  Based 
Reconnaissance  Satellites  (SBRS)  for  the  U.  S.  Air  Force,  and  2)  Fleet 
Ballistic  Missile  (FBM)  for  the  U.S.  Navy’s  Trident  class  ballistic  missile 
submarines  (SSBN).  This  facility  also  produces  satellites  for  the  National 
Reconnaissance  Office  (NRO). 

Z  Contract  organization  fit  within  the  organization.  Within  the  Lockheed 
Martin  organization  at  this  facility,  the  contracting  organization  fits  in  a 
Staff  position  reporting  directly  to  the  VP  of  Strategic  Missile  Programs. 

Section  II— PPI  COLLECTION 

3.  Is  PPI  collected  on  each  contractor/firm  prior  to  each  contract?  Yes,  on 
all  requirements 


a.  Business  Relations: 

Yes 

b.  Quality: 

Yes 

c.  Schedule: 

Yes 

d.  Cost  Control: 

Yes 

Manager,  Strategic  Missile  Procurement  comments:  The  performance  data  is 
typically  reviewed  quarterly  in  the  form  of  Program  Management  Reviews  (PMR) 
with  the  subcontractor. 


For  routine  purchases: 

a.  Business  Relations: 

No 

b.  Quality: 

Yes 

c.  Schedule: 

Yes 

d.  Cost  Control: 

No 

Manager,  Strategic  Missile  Procurement  comments:  Quality  and  schedule  data  is 
gathered  on  ALL  procurements,  regardless  of  dollar  value. 


4.  PPI  data  management. 

a.  How  is  PPI  data  collected:  Supplier  Report  Card  (Scorecard) 

b.  How  is  PPI  data  managed  and  available:  Automated  database  via 
password  security.  Delivery  and  quality  is  tracked  on  ALL  incoming  material 
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from  subcontractors.  Deliveries  are  bar-code  scanned  at  the  receiving  area.  Late 
deliveries  are  noted.  Adjustments  can  be  made  if  agreed  between  prime  and  sub. 
Quality  is  tracked  via  Quality  Deficiency  Reporting  (QDR).  QDRs  are  assigned  to 
each  subcontractor.  QDRs  and  late  deliveries  deduct  from  the  contractor’s 
overall  score  based  on  a  100-point  system.  A  printout  of  vendor  history  is 
available  on  demand.  Suppliers  are  given  opportunity  to  rebut  findings. 

5.  Request  for  Proposal  (RFP)  preparation. 

a.  Is  PPI  data  requested  in  each  RFP?  Yes,  the  RFP  is  more  specific  and 

spells  out  the  factors  and  range  of  data  that  will  be  considered. 

-  Relevancy  (similar  in  scope):  Yes 

-  Currency  (<3  yrs):  Yes  (</=  3  years) 

-  Quality  (good  performance):  Yes 

-  Experience  (#  of  years  in  the  ind):  Yes 

-  Other  factors:  Yes  (check  Dunn  &  Bradstreet) 

b.  Do  you  stipulate  intention  to  conduct  research  of  named  references?  Yes 

-  Telephone  interviews:  Yes 

-  Questionnaires:  No 

c.  Do  you  stipulate  intention  to  use  other  than  named  references? 

-  End  users:  Yes  (sister  divisions) 

-  Other  customers:  Yes 

-  Passive  PPI  (i.e.  QDRs):  Yes 

-  Site  visits:  N/A 

-  Pre-award  surveys:  Yes  (see  comments) 

Manager,  Strategic  Missile  Procurement  Comments:  If  nothing  is  known  about  a 
company,  a  team  is  sent  out  to  that  company  consisting  of:  1)  engineering,  2) 
quality  assurance,  3)  contracting  rep,  and  4)  a  costing  specialist. 

Researcher  comments:  The  Supplier  Report  Card  appears  to  have  same 
characteristics  of  the  U.  S.  Navy’s  Red,  Yellow  and  Green  program. 

6.  Discuss  your  experience  with  CPARS  as  a  source  of  PPI:  N/A 


7.  If  not  CPARS,  what  sources  (or  industry  model)  do  you  use  to  gather  and 
maintain  PPI  on  a  contractor  or  firm? 

a.  Is  your  system  streamlined  (user  friendly)?  YES,  Supplier  Report  Card 
Automated?  YES 
Web-based?  NO 

Kept  current?  YES.  Closed-loop  system  between  Contracts/Purchasing 
and  Receiving  and  Quality  Control. 

Recorded  for  each  contract  at  least  annually,  interim,  and  final?  YES,  All 
procurements. 

Contractor  given  opportunity  to  comment  on  PPI  appraisals?  YES 
Performance  assessments  discussed  during  initial  post  award  conference? 

NO,  quarterly  letter  tells  each  contractor  what  their  score  is.  Also 
comes  with  graphs. 


b. 

c. 

d. 


f. 

g- 
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SECTION  3:  PAST  PERFORMANCE  INFORMATION— EVALUATION  (INITIAL 
PROCUREMENT) 


8.  Who  typically  comprises  your  acquisition  team  for  an  initial  procurement  of 
estimated  amount  (e.q.  $1M  or  more)? 


a.  Procuring  Contracting  Officer: 

b.  Legal  counsel: 

d.  Technical  Evaluation  Board: 

e.  Past  Performance  Board: 

Separate  (Contracting  AND  Program 

f.  Cost/Price  Analyst: 

g.  End  User: 


h.  Engineering  Rep: 

i.  Logistics  Rep: 

j.  Comptroller/Budget: 

k.  Small  Business  Rep: 

l.  Quality  Assurance  Rep: 


_X_ 

_ Oust  advice) 

_ X _ (systems  integration  &  test) 

_X_ 

Office  reps) 

X 

_ X _ (responsible  equip  engineer 

who  has  technical  responsibility  for 
the  item  or  equipment) 

X 


X 


9.  Is  PPI  a  factor  (or  sub-factor)  in  your  initial  procurements?  YES 

a.  Resident  in  your  Request  for  Proposal  (RFP)?  Yes  X  No _ 

b.  If  yes,  what  is  the  relative  weight  compared  to  other  source  selection  factors? 

□  4.  D3.  X  2.  □  1 . 

Highest  Weight  Lowest  Weight 

c.  In  a  Best  Value-tradeoff  approach,  how  do  you  typically  rank  past 
performance  relative  to  other  factors? 


1)  Technical  Capability  (Most  important) 

2)  Cost/Price 

3)  Past  Performance 

4)  Other  factors  (Least  important) 


10.  Has  Past  Performance  of  a  contractor  or  firm  ever  been  the  deciding  factor 
for  elimination  from  the  competitive  range  or  non-selection  for  contract  award?  NO 

a.  Have  legal  recourse/challenges  been  experienced  IRT  use  of  PPI  as  a  basis 

for  non-selection?  N/A 

b.  If  so,  did  your  organization  provide  satisfactory  justification  for  its  decision  not 

to  select  a  contractor  or  firm?  N/A 


1 1 .  Does  your  organization  have  an  established  list  of  favorite  “go-to” 
contractors  or  firms?  There  is  no  “go-to”  list  for  procurements.  Only  a  well 
documented  sole-source  supplier. 

a.  How  does  a  contractor  or  firm  get  on  the  list?  As  a  sole  source 

b.  How  does  a  contractor  or  firm  stay  on  the  list?  Stay  a  sole  source 
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c.  How  does  a  contractor  or  firm  get  removed  from  the  list  ?  Poor 
performance 

d.  Can  a  contractor  or  firm  continue  making  offers  (bids/proposals)  after  being 
removed  from  the  go-to  list  OR  are  they  out  forever?  The  offending  contractor  or  firm 
would  not  get  an  RFP. 

e.  Does  your  firm  use  PPI  data  as: 

i.  Comparative  (relevant,  current,  guality) 

OR  only  as 

ii.  Pass/Fail  (i.e.  on  Suspension,  debarment)  N/A 


Procurement  Manager  comments.  Look  at  report  card  data  early  in  the 
process. 


12.  RFP  evaluation  factors — PPI  data  evaluation: 

YES 


YES 

YES  (3-5  yrs) 

YES 

YES 

No 


a.  PPI  data  reguested  in  RFP? 

-  Relevancy  (similar  in  scope): 

-  Currency  (<3  yrs): 

-  Quality  (good  performance): 

-  Experience  (#  of  years  in  the  ind) 

-  Other  factors: 

b.  Stipulate  intention  to  conduct  research  of  named  references? 

b.  Telephone  interviews:  YES 

c.  Questionnaires:  No 

c.  Stipulate  intention  to  use  other  than  named  references? 

d.  End  users:  YES  (sister  divisions) 

e.  Other  customers:  No 

f.  Passive  PPI  (i.e.  QDRs):  YES 

g.  Site  visits:  N/A 

h.  Pre-award  surveys:  YES 


Manager,  Strategic  Missile  Procurement  comments:  1)  Protests.  Lockheed  Martin 
(LM),  as  a  private-sector  entity,  does  not  have  the  same  level  of  protest  risk  from 
disappointed  bidders  as  contracting  activities  in  the  Public  (Government)  Sector. 
However,  a  disappointed  bidder  has  the  option  of  calling  his/her  congressman  or 
senator.  LM  devotes  much  time  and  resources  to  answering  congressionals  and 
defending  contracting  awards.  2)  Oversight.  As  a  government  contractor,  LM 
also  devotes  resources  to  audits  from  Defense  Contract  Audit  Agency  (DCAA) 
and  surveillance  from  Defense  Contract  Management  Agency  (DCMA). 
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APPENDIX  8:  Private  Sector  (Defense-focused) 
Activity  #2:  Northrop  Grumman,  Newport  News 

SECTION  I— GENERAL 


1 .  Organization  Mission.  At  Northrop  Grumman,  Newport  News,  located  in 
Newport  News,  VA,  the  product  lines  are:  1)  U.S.  Navy  shipbuilding,  conversion 
and  overhauls,  and  2)  Commercial  ship  repairs  and  dry-docking.  The  vast 
majority  of  work  performed  is  for  the  U.S.  Navy  (over  90%). 

2.  Contract  organization  fit  within  the  organization.  Within  the  Northrop 
Grumman  organization  at  this  facility,  the  contracting  organization  fits  in  the  2nd 
tier  position  under  the  Executive  VP,  Supply  Chain  Management,  reporting 
directly  to  the  Senior  VP  of  Operations. 

SECTION  II— PPI  COLLECTION 

3.  Is  PPI  collected  on  each  contractor/firm  prior  to  each  contract?  Yes,  using 
Scorecard  methodology,  for  all  material  requirements  (service  suppliers  do  not 
receive  a  scorecard  at  this  time). 

a.  Business  Relations:  Yes  (responsiveness  to  implement 

corrective  actions) 

b.  Quality:  Yes 

c.  Schedule:  Yes 

d.  Cost  Control:  Yes 


Manager,  Submarine  &  Carrier  Procurement  &  Project  Engineer,  Strategic  Sourcing 
comments:  The  performance  data  is  reviewed  as  often  as  quarterly  or 
semiannually  in  the  form  of  a  score  card  (lOOpt  scale)  and  letter  sent 
electronically  or  via  the  postal  system. 


4.  PPI  data  management. 

a.  How  is  PPI  data  collected:  Supplier  Report  Card.  An  approximate  total  of 
2400  suppliers  do  business  with  Northrop  Grumman  Newport  News.  Currently 
the  top  ~  400  +  suppliers  are  tracked  via  scorecards  due  to  annual  volume  of 
orders. 

b.  How  is  PPI  data  managed  and  available:  Same  as  Lockheed,  an  automated 
intranet  database  with  password  security.  Delivery  and  quality  are  tracked  on 
ALL  incoming  material  from  subcontractors.  Deliveries  are  bar-code  scanned  at 
the  receiving  area.  Late  deliveries  are  noted.  Adjustments  can  be  made  if  agreed 
between  prime  and  sub.  Quality  is  tracked  via  Quality  Deficiency  Reporting 
(QDR)-type  system.  QDRs  and  late  deliveries  deduct  from  the  contractor’s  overall 
score  based  on  a  100-point  scorecard  system.  A  printout  of  vendor  history  is 
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available  on  demand.  Suppliers  are  quick  to  rebut  findings.  Program  is  similar 
to  the  Navy’s  Red,  Yellow,  Green  system  in  that  it  uses  a  closed-loop  supply- 
chain  system  with  QC  inspectors  co-located  at  receiving  points. 


Yes 

Yes 

Yes  (</=  3  years) 

Yes 

Not  really,  only  if  SB  startup 
Yes  (check  stability  of  firm) 


5. Request  for  Proposal  (RFP)  preparation, 

a.  Is  PPI  data  requested  in  each  RFP? 

-  Relevancy  (similar  in  scope): 

-  Currency  (<3  yrs): 

-  Quality  (good  performance): 

-  Experience  (#  of  years  in  the  ind): 

-  Other  factors: 

c.  Do  you  stipulate  intention  to  conduct  research  of  named  references?  Yes 

-  Telephone  interviews:  Yes 

-  Questionnaires:  Yes  (2  page  questionnaire, 

EOE,  safety,  quality  etc.) 

c.  Do  you  stipulate  intention  to  use  other  than  named  references? 

-  End  users:  Yes  (sister  sector,  business 

partners) 

-  Other  customers:  Yes  (EB  etc.) 

-  Passive  PPI  (i.e.  QDRs):  Yes 

-  Site  visits:  N/A 

-  Pre-award  surveys:  Yes  (see  comments) 

Manager,  Submarine  &  Carrier  Procurement  &  Project  Engineer,  Strategic  Sourcing 
comments:  Pre-award  surveys  are  used  if  little  is  known  about  a  supplier  and  the 
award  is  expected  to  exceed  $100K,  or  a  long-term  relationship  will  exist. 


6.  Discuss  your  experience  with  CPARS  as  a  source  of  PPI:  N/A 


7.  If  not  CPARS,  what  sources  (or  industry  model)  do  you  use  to  gather  and 
maintain  PPI  on  a  contractor  or  firm? 

a.  Is  your  system  streamlined  (user  friendly)?  YES,  Supplier  Report  Card 

b.  Automated?  YES 

c.  Web-based?  NO  (intranet  only)  shared  with 

sister  locations  and  industry 
partners  only. 

d.  Kept  current?  YES,  closed-loop  system. 

e.  Recorded  for  each  contract  at  least  annually,  interim,  and  final? 

YES,  All  procurements. 

f.  Contractor  given  opportunity  to  comment  on  PPI  appraisals?  YES 

g.  Performance  assessments  discussed  during  initial  post  award  conference? 

NO,  quarterly  letter  tells  each  contractor  what  their  score  is. 

Buyer/Supplier  post  award  discussions  are  entertained  if  requested  but  not 
common  practice. 
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SECTION  3:  PAST  PERFORMANCE  INFORMATION— EVALUATION  (INITIAL 
PROCUREMENT) 


8.  Who  typically  comprises  your  acquisition  team  for  an  initial  procurement  of 
estimated  amount  (e.q.  $1M  or  more)? 


a. 

b. 

d. 

e. 

f. 

g- 


Procuring  Contracting  Officer:  X _ 

Legal  counsel:  X _ (Contracts  &  Legal) 

Technical  Evaluation  Board:  _ X _ (systems  integration  &  test) 

Past  Performance  Board:  X _ 

-  Separate  (Contracting  AND  Program  Office  reps) 

Cost/Price  Analyst:  X _ 

End  User:  X _ (responsible  equip  engineer 

who  has  technical  responsibility  for 
the  item  or  equipment) 


i.  Engineering  Rep:  X 

j.  Logistics  Rep:  _ 

k.  Comptroller/Budget:  _ 

l.  Small  Business  Rep:  X 

m.  Quality  Assurance  Rep:  X 


9.  Is  PPI  a  factor  (or  sub-factor)  in  your  initial  procurements?  YES  for 

procurements  valued  at  or  above  $25K 

a.  Resident  in  your  Request  for  Proposal  (RFP)?  Yes  X  No _ 

b.  If  yes,  what  is  the  relative  weight  compared  to  other  source  selection  factors? 

□  4.  □  3.  D  2.  □  1 . 

Total  Value  concept:  all  equally  weighted  (proposals  must  first  be  technically 
acceptable).  Every  procurement,  however,  can  take  on  a  different  ranking  of 
factors. 

Highest  Weight  Lowest  Weight 

c.  In  a  Best  Value-tradeoff  approach,  how  do  you  typically  rank  past 
performance  relative  to  other  factors? 


1 )  Technical  Capability  (Most  important) 

2)  Cost/Price 

3)  Past  Performance 

4)  Other  factors  (Least  important) 

NG  only  solicits  (sends  FRS  to  contractors  in  which  they  have  a  good  working 
relationships).  NG  enjoys  a  robust  vendor  base  of  approx.  2400  suppliers. 
Approximately  400  of  these  suppliers  have  a  quarterly  thru-put  to  warrant  a 
scorecard. 


10.  Has  Past  Performance  of  a  contractor  or  firm  ever  been  the  deciding  factor 
for  elimination  from  the  competitive  range  or  non-selection  for  contract  award?  YES, 
due  to  delivery.  Supplier  developed  a  history  of  poor  delivery  performance. 
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a.  Have  legal  recourse/challenges  been  experienced  IRT  use  of  PPI  as  a  basis 

for  non-selection?  Yes.  (Congressional,  both) 

i.  Supplier  contacted  the  office  of  Senator  Simons.  1993. 

ii.  Supplier  contacted  the  office  of  Senator  Warner.  Year  not 
available. 

b.  If  so,  did  your  organization  provide  satisfactory  justification  for  its  decision 
not  to  select  a  contractor  or  firm?  Yes.  Senators  typically  call  in  order  to 
“inquire”  as  to  what  has  happened.  In  both  inquires,  NG  pulled  together 
the  facts  via  top  management,  presented  findings,  and  the  issues  were 
resolved. 


1 1 .  Does  your  organization  have  an  established  list  of  favorite  “go-to” 

contractors  or  firms?  Yes.  It  is  standard  practice  to 

go  to  contractors  with  good  scorecard  performance.  NG,  being  private  sector  is 
not  obligated  to  practice  “full  and  open”  competition.  Competition  is  sought 
through  the  use  of  best  value  (Total  Value)  criteria. 

a.  How  does  a  contractor  or  firm  get  on  the  list?  Usually  must  be  a  SB  startup 
AND  NG  is  looking  to  expand  their  vendor  base  in  an  area  of  the  SB’s  specialty  or 
technical  expertise/superiority  designed  into  product. 

b.  How  does  a  contractor  or  firm  stay  on  the  list?  Good  score,  continuous 
performance  AND  evidence  of  added  value  (i.e.  3PL  implementation,  just-in-time 
(JIT)  processes  etc.) 

c.  How  does  a  contractor  or  firm  get  removed  from  the  list  ?  Poor  Scorecard 
performance  and  lack  of  value-added  services. 

d.  Can  a  contractor  or  firm  continue  making  offers  (bids/proposals)  after  being 
removed  from  the  go-to  list  OR  are  they  out  forever?  Out  forever  unless  NG  needs  to 
expand  vendor  base  AND/OR  the  offending  contractor  or  firm  can  demonstrate 
positive  changes  within  their  organization.  Contractor  must  demonstrate 
something  has  changed  substantially  to  be  reconsidered. 

e.  Does  your  firm  use  PPI  data  as: 

i.  Comparative  (relevant,  current,  quality) 

OR  only  as 

ii.  Pass/Fail  (i.e.  on  Suspension,  debarment)  BOTH. 

12.  RFP  evaluation  factors — PPI  data  evaluation: 


a.  PPI  data  requested  in  RFP? 

-  Relevancy  (similar  in  scope): 

-  Currency  (<3  yrs): 

-  Quality  (good  performance): 

-  Experience  (#  of  years  in  the 

-  Other  factors: 


YES 

YES 

YES 

YES 

.):  YES  (unless  supplier  is  a 

startup) 

No 
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b.  Stipulate  intention  to  conduct  research  of  named  references? 

-  Telephone  interviews:  YES 

-  Questionnaires:  YES 

c.  Stipulate  intention  to  use  other  than  named  references? 

-  End  users:  YES  (sister  divisions: 

Avondale/Ingalls,  General 
Dynamics  etc.) 

-  Other  customers:  YES 

-  Passive  PPI  (i.e.  QDRs):  YES  (based  on  Scorecard) 

-  Site  visits:  N/A 

-  Pre-award  surveys:  YES  (if  new  supplier) 
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APPENDIX  9:  PRIVATE  SECTOR  (NON-DEFENSE- 
FOCUSED)  ACTIVITY  #1:  Opalsoft 


Section  I— GENERAL 

1 .  Organization  Mission.  At  Opal  Soft  located  in  San  Jose,  CA,  the  product 
lines  are:  1)  Computer  Design  and  Development  software,  and  2)  System 
Integration  &  Enterprise  Resource  Planning.  Opal  Soft  is  registered  as  a  small 
disadvantaged  business  and  is  currently  in  the  8A  program.  Opal  Soft  was  also 
awarded  a  GSA  schedule  for  IT  professional  services. 

2.  Contract  organization  fit  within  the  organization.  Within  Opal  Soft,  the 
contracting  organization  fits  in  the  Staff  position  as  Director,  Business 
Development  and  reports  directly  to  the  president  of  Opal  Soft. 

Section  II— PPI  COLLECTION 

3.  Is  PPI  collected  on  each  contractor/firm  prior  to  each  contract?  Yes,  on 
all  requirements. 


a.  Business  Relations: 

Yes 

b.  Ouality: 

Yes 

c.  Schedule: 

Yes 

d.  Cost  Control: 

Yes 

Director,  Business  Development  comments:  The  performance  data  is  reviewed  as 
often  a  quarterly  in  the  form  of  a  Program  Management  Review  (PMR)  with  the 
subcontractor. 

Routine  purchases: 

a.  Business  Relations 

b.  Ouality: 

c.  Schedule: 

d.  Cost  Control: 

4.  PPI  data  management. 

a.  How  is  PPI  data  collected:  Manual  system,  kept  in  supplier’s  file. 

b.  How  is  PPI  data  managed  and  available:  Kept  in  supplier’s  file.  Not 
automated. 


No 

Yes 

Yes 

No 


5.  Reguest  for  Proposal  (RFP)  preparation. 

a.  Is  PPI  data  requested  in  each  RFP?  No.  Typically  outsourcing  of 

services  or  procurement  of  hardware  is  less  formal. 

-  Relevancy  (similar  in  scope):  N/A 
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-  Currency  (<3  yrs):  N/A 

-  Quality  (good  performance):  N/A 

-  Experience  (#  of  years  in  the  ind):  N/A 

-  Other  factors:  Yes  (check  Dun  &  Bradstreet) 

b.  Do  you  stipulate  intention  to  conduct  research  of  named  references?  Yes 

-  Telephone  interviews:  Yes  (word  of  mouth, 

networking  at  trade  shows  & 
conventions) 

-  Questionnaires:  No 

c.  Do  you  stipulate  intention  to  use  other  than  named  references? 

-  End  users:  Yes  (Technology  partners) 

-  Other  customers:  No 

-  Passive  PPI  (i.e.  QDRs):  No 

-  Site  visits:  N/A 

-  Pre-award  surveys:  Yes  (see  comments) 

Director,  Business  Development  comments:  On  occasion,  Opal  Soft  will  outsource  a 
hardware  requirement  in  order  to  meet  a  scheduled  event  but  primarily  performs 
all  contract  requirements  in-house  with  very  few  exceptions. 


6.  Discuss  your  experience  with  CPARS  as  a  source  of  PPI:  N/A 


7.  If  not  CPARS,  what  sources  (or  industry  model)  do  you  use  to  gather  and 
maintain  PPI  on  a  contractor  or  firm? 

a.  Is  your  system  streamlined  (user  friendly)?  NO,  system  is  manual  and  is 

documented  in  each  vendor’s 
contract  file. 

b.  Automated?  NO 

c.  Web-based?  NO 

d.  Kept  current?  YES,  manual  system. 

e.  Recorded  for  each  contract  at  least  annually,  interim,  and  final?  NO,  only 
good  or  bad  past  performance  is  documented  (e.g.  legal  issues,  late 
payments  etc.) 

f.  Contractor  given  opportunity  to  comment  on  PPI  appraisals?  YES 

g.  Performance  assessments  discussed  during  initial  post  award  conference? 


NO 


SECTION  3:  PAST  PERFORMANCE  INFORMATION— EVALUATION  (INITIAL 
PROCUREMENT) 


8.  Who  typically  comprises  your  acquisition  team  for  an  initial 
procurement  of  estimated  amount  (e.g.  $1M  or  more)? 


a.  Procuring  Contracting  Officer: 

c.  Legal  counsel: 

d.  Technical  Evaluation  Board: 


X_ 

_ (no  review,  legal  counsel  held 

on  retainer) 

X _ (domain  expert) 
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e.  Past  Performance  Board:  X _ 

a.  Separate  (2  program  or  project  managers) 

f.  Cost/Price  Analyst:  X _ 

g.  End  User:  _ 

h.  Engineering  Rep:  X _ 

i.  Logistics  Rep:  _ 

j.  Comptroller/Budget:  X _ (accounts  manager) 

k.  Small  Business  Rep:  _ 

l.  Quality  Assurance  Rep:  _ 

9.  Is  PPI  a  factor  (or  sub-factor)  in  your  initial  procurements?  YES 

m.  Resident  in  your  Request  for  Proposal  (RFP)?  Yes  X  No _ 

n.  If  yes,  what  is  the  relative  weight  compared  to  other  source  selection  factors? 

X  4.  D3.  D2.  D  1. 

Highest  Weight  Lowest  Weight 

o.  In  a  Best  Value-tradeoff  approach,  how  do  you  typically  rank  past 
performance  relative  to  other  factors? 

1)  Past  Performance  (Most  important) 

2)  Technical  Capability 

3)  Cost/Price 

4)  Other  factors  (Least  important) 

10.  Has  Past  Performance  of  a  contractor  or  firm  ever  been  the  deciding  factor 
for  elimination  from  the  competitive  range  or  non-selection  for  contract  award?  YES 

a.  Have  legal  recourse/challenges  been  experienced  IRT  use  of  PPI  as  a  basis 

for  non-selection?  NO 

b.  If  so,  did  your  organization  provide  satisfactory  justification  for  its  decision  not 

to  select  a  contractor  or  firm?  N/A 

Director,  Business  Development  comments:  Opal  has  succeeded  in  using  past 
performance  as  a  primary  reason  for  non-selection.  It  has  not  been  challenged, 
potentially  (management  believes)  because  Opal  is  not  a  large  company.  Opal,  as 
a  subcontractor,  placing  a  bid  with  a  prime  contractor  has  determined  that  further 
outsourcing  to  a  capable,  but  unreliable,  vendor  is  too  risky  for  critical  path 
supplies  or  services. 

1 1 .  Does  your  organization  have  an  established  list  of  favorite  “go-to” 

contractors  or  firms?  YES 

a.  How  does  a  contractor  or  firm  get  on  the  list?  By  reference.  Vendors  make 
contact  (network)  at  GSA  conventions,  SB  monthly  meetings  etc. 

b.  How  does  a  contractor  or  firm  stay  on  the  list?  Good  &  reliable 
performance 

c.  How  does  a  contractor  or  firm  get  removed  from  the  list?  Poor  performance 
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d.  Can  a  contractor  or  firm  continue  making  offers  (bids/proposals)  after  being 
removed  from  the  go-to  list  OR  are  they  out  forever?  No,  they  are  more  or  less  out 
forever. 


e.  Does  your  firm  use  PPI  data  as: 

i.  Comparative  (relevant,  current,  quality)  use  of  resumes 

a.  OR  only  as 

ii.  Pass/Fail  (i.e.  on  Suspension,  debarment)  N/A 
12.  RFP  evaluation  factors — PPI  data  evaluation: 


a.  PPI  data  requested  in  RFP? 


-  Relevancy  (similar  in  scope): 

-  Currency  (<3  yrs): 

-  Quality  (good  performance): 

-  Experience  (#  of  years  in  the  ind) 


-  Other  factors: 

b.  Stipulate  intention  to  conduct  research  of  named  references? 

-  Telephone  interviews:  YES 

-  Questionnaires:  No 

c.  Stipulate  intention  to  use  other  than  named  references? 

-  End  users:  No 

-  Other  customers:  No 

-  Passive  PPI  (i.e.  QDRs):  YES 

-  Site  visits:  N/A 

-  Pre-award  surveys:  No 


No  official  RFP  used;  however, 

data  required  for  hiring 

(outsourcing)  a  requirement  is 

similar 

YES 

YES 

YES 

YES  (if  they  survived  the  .COM 
bust,  then  they  may  have 
something  to  offer) 

No 
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APPENDIX  10:  PRIVATE  SECTOR  (NON-DEFENSE- 
FOCUSED)  ACTIVITY  #2:  Maersk  Line,  Limited 


Section  I— GENERAL 

1.  Organization  Mission.  At  Maersk  Line,  Limited  (MLL),  located  in 
Norfolk,  VA,  the  product  line  is  the  delivery  of  global  ship  owning  &  management 
service,  transportation,  and  logistic  solutions  information  technology 
development,  and  implementation  services  to  our  U.  S.  government  and 
commercial  customers.  MLL  owns  or  operates  50  US  Flag  vessels. 

2.  Contract  organization  fit  within  the  organization.  Within  MLL,  the 
contracting  organization  fits  in  the  organization  as  a  branch  in  the  2nd  tier  under 
Legal  Affairs  &  Contracting  department.  MLL  is  led  by  a  Chief  Executive  Officer. 

Section  II— PPI  COLLECTION 


3.  Is  PPI  collected  on  each  contractor/firm  prior  to  each  contract?  Yes,  on 
all  requirements 


a.  Business  Relations: 

b.  Quality: 

c.  Schedule: 

d.  Cost  Control: 


Yes 

Yes 

Yes  (on-time  delivery  is  paramount) 
Yes 


Routine  purchases: 


a.  Business  Relations:  No 

b.  Quality:  Yes 

c.  Schedule:  Yes 

d.  Cost  Control:  No 


4.  PPI  data  management. 

a.  How  is  PPI  data  collected:  Intranet  (Peoplesoft)  for  accounting,  finance, 
purchasing  +  manual  supplier  files  kept  by  the  port  engineer  along  with  any 
quality  deficiency  reports  (QDRs) 

b.  How  is  PPI  data  managed  and  available:  Kept  in  supplier’s  file.  Partially 
automated. 


5.  Request  for  Proposal  (RFP)  preparation. 

a.  Is  PPI  data  requested  in  each  RFP?  Yes,  the  RFP  is  more  specific  and 
spells  out  the  factors  and  range  of  data  that  will  be  considered. 

-  Relevancy  (similar  in  scope):  Yes 

-  Currency  (<3  yrs):  Yes  (</=  3  years) 
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-  Quality  (good  performance):  Yes 

-  Experience  (#  of  years  in  the  ind):  No 

-  Other  factors:  No 

b.  Do  you  stipulate  intention  to  conduct  research  of  named  references?  Yes 

-  Telephone  interviews:  No 

-  Questionnaires:  Yes 

c.  Do  you  stipulate  intention  to  use  other  than  named  references? 

-  End  users:  Yes  (via  other  agency 

agreements) 

-  Other  customers:  Yes  (Husbanding  Agents) 

-  Passive  PPI  (i.e.  QDRs):  Yes 

-  Site  visits:  N/A 

-  Pre-award  surveys:  Yes  (See  comments) 

Director  of  Contracts  and  Contract  Administrator  comments:  On  occasion  MLL  may 
send  a  team  (e.g.  Port  Engineer,  Contracts  Rep  etc.)  to  the  site  in  order  to  verify 
or  clarify  ambiguous  statements  made  in  an  offeror’s  bid  or  proposal. 

6.  Discuss  your  experience  with  CPARS  as  a  source  of  PPI:  N/A 

7.  If  not  CPARS,  what  sources  (or  industry  model)  do  you  use  to  gather  and 

maintain  PPI  on  a  contractor  or  firm?  Manual  system 

a.  Is  your  system  streamlined  (user  friendly)?  Partially  automated.  Intranet 
(Peoplesoft)  for  accounting,  finance,  purchasing  +  manual  supplier  files  kept  buy 
the  port  engineer  along  with  any  quality  deficiency  reports  (QDRs). 

b.  Automated?  Partially  automated. 

c.  Web-based?  NO 

d.  Kept  current?  YES,  automated  +manual 

system. 

e.  Recorded  for  each  contract  at  least  annually,  interim,  and  final? 

NO,  only  good  or  bad  past  performance  is  documented  (e.g.  legal  issues,  late 
payments,  etc.) 

f.  Contractor  given  opportunity  to  comment  on  PPI  appraisals? 

YES,  informal  process  (during  production  meetings) 

g.  Performance  assessments  discussed  during  initial  post  award  conference? 

Purchasing:  No.  Large  Contracts:  Yes.  Performance  assessments  are 
more  or  less  informal  in  the  forum  of  production  meetings  via  MLL’s  Port 
Engineer  and  the  sub-contractor. 
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SECTION  3:  PAST  PERFORMANCE  INFORMATION— EVALUATION  (INITIAL 
PROCUREMENT) 


8.  Who  typically  comprises  your  acquisition  team  for  an  initial  procurement  of 
estimated  amount  (e.q.  $1M  or  more)? 

a.  Procuring  Contracting  Officer: 


b.  Legal  counsel: 

c.  Technical  Evaluation  Board: 

d.  Past  Performance  Board: 

e.  Cost/Price  Analyst: 

f.  End  User/Engineering  Rep: 

g.  Logistics  Rep: 

h.  Comptroller/Budget: 

i.  Small  Business  Rep: 

j.  Quality  Assurance  Rep: 


X _ (Director  of  Contracts  + 

Contracts  Administrator) 

_ (Recently  hired  Legal  Counsel) 

X _ (picks  ~  five  people) 

X 

X_ 

X _ (Port  Engineer) 

X 


X 


9.  Is  PPI  a  factor  (or  sub-factor)  in  your  initial  procurements?  YES 

a.  Resident  in  your  Request  for  Proposal  (RFP)?  Yes  X  No _ 

b.  If  yes,  what  is  the  relative  weight  compared  to  other  source  selection 
factors? 

□  4.  3.X . X  2.  I  1. 

Technical  Past  Performance  =  Price 

Notional  25%  weighting 

Highest  Weight  Lowest  Weight 

c.  In  a  Best  Value-tradeoff  approach,  how  do  you  typically  rank  past 
performance  relative  to  other  factors? 

1)  Technical  Capability  (most  important) 

2)  Cost/Price  AND  Past  Performance 

10.  Has  Past  Performance  of  a  contractor  or  firm  ever  been  the  deciding  factor 
for  elimination  from  the  competitive  range  or  non-selection  for  contract  award?  NO 

a.  Have  legal  recourse/challenges  been  experienced  IRT  use  of  PPI  as  a  basis 
for  non-selection?  NO 

b.  If  so,  did  your  organization  provide  satisfactory  justification  for  its  decision  not 

to  select  a  contractor  or  firm?  N/A 
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1 1 .  Does  your  organization  have  an  established  list  of  favorite  “go-to” 
contractors  or  firms?  YES,  1)  for  dry-docking,  the  east  (or  west)  coast  dry-docks 
and  their  dry-dock  capabilities  are  already  well  known,  2)  for  equipment  &  spares, 
MLL  has  an  approved  vendor  database. 

a.  How  does  a  contractor  or  firm  get  on  the  list?  Vendor  completes  a  business 
classification  profile  form,  forwards  their  line  cards  to  MLL  buyers. 

b.  How  does  a  contractor  or  firm  stay  on  the  list?  Good  &  reliable 
performance. 

c.  How  does  a  contractor  or  firm  get  removed  from  the  list?  Poor 
performance.  Vendor  is  flagged  in  Peoplesoft  and  put  on  suspension. 

d.  Can  a  contractor  or  firm  continue  making  offers  (bids/proposals)  after  being 
removed  from  the  go-to  list  OR  are  they  out  forever?  No,  they  are  more  or  less  out 
unless  the  respective  vendor  can  prove  that  improvements  have  been  made. 

e.  Does  your  firm  use  PPI  data  as: 

Comparative  (relevant,  current,  quality)  YES 
OR  only  as 

Pass/Fail  (i.e.  on  Suspension,  debarment)  YES,  check  of 
government  suspended/debarred  list  (flagged  in  “favorites”) 

12.  RFP  evaluation  factors — PPI  data  evaluation: 

a.  PPI  data  requested  in  RFP?  No  official  RFP  used,  however,  data  required 


for  hiring  (outsourcing)  a  requirement  is  similar. 

-  Relevancy  (similar  in  scope):  YES 

-  Currency  (<3  yrs):  YES 

-  Quality  (good  performance):  YES 

-  Experience  (#  of  years  in  the  ind):  YES 

-  Other  factors:  No 

b.  Stipulate  intention  to  conduct  research  of  named  references? 

-  Telephone  interviews:  No 

-  Questionnaires:  YES 

c.  Stipulate  intention  to  use  other  than  named  references? 

-  End  users:  YES 

-  Other  customers:  YES 

-  Passive  PPI  (i.e.  QDRs):  YES 

-  Site  visits:  N/A 

-  Pre-award  surveys:  YES 
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Initial  Distribution  List 


1.  Defense  Technical  Information  Center  2 

8725  John  J.  Kingman  Rd„  STE  0944;  Ft.  Belvoir,  VA  22060-6218 

2.  Dudley  Knox  Library,  Code  013  2 

Naval  Postgraduate  School,  Monterey,  CA  93943-5100 

3.  Research  Office,  Code  09  1 

Naval  Postgraduate  School,  Monterey,  CA  93943-5138 

4.  Douglas  A.  Brook  1 

Dean,  GB/Kb 

555  Dyer  Road,  Naval  Postgraduate  School,  Monterey,  CA  93943-5000 

5.  Keith  F.  Snider  1 

Associate  Professor,  GB/Sk 

555  Dyer  Road,  Naval  Postgraduate  School,  Monterey,  CA  93943-5000 

6.  James  B.  Greene  1 

Acquisition  Chair,  GB/Jg 

555  Dyer  Road,  Naval  Postgraduate  School,  Monterey,  CA  93943-5000 

7.  Bill  Gates  1 

Associate  Dean  for  Research,  GB/Gt 

555  Dyer  Road,  Naval  Postgraduate  School,  Monterey,  CA  93943-5000 

8.  CDR  Roger  Lord,  SC,  USN  4 

Title,  GB/Lr 

555  Dyer  Road,  Naval  Postgraduate  School,  Monterey,  CA  93943-5000 

9.  Karey  L.  Shaffer  1 

Program  Manager,  Acquisition  Research  Program,  GB/Ks 

555  Dyer  Road,  Naval  Postgraduate  School,  Monterey,  CA  93943-5000 


Copies  of  the  Acquisition  Sponsored  Research  Reports  may  be  printed  from  our 
website  www.nps.navy.mil/qsbpp/acqn/publications 
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FY  2005  Sponsored  Acquisition  Research  Products 

Working  Paper  Series 

NPS-AM-05-001  Engagement  versus  Disengagement: 


NPS-AM-05-002 

How  Structural  &  Commercially-Based  Regulatory  Changes  have 
Increased  Government  Risks  in  Federal  Acquisitions 

November  2004 

The  Yoder  Three-Tier  Model  for  Optimizing  Contingency 
Contracting  Planning  and  Execution 

December  2004 

FY  2004  Sponsored  Acquisition  Research  Products 

Sponsored  Report  Series 

NPS-CM-04-019  Contractor  Past  Performance  Information  (PPI)  In  Source 


NPS-AM-04-013 

Selection:  A  comparison  Study  of  Public  and  Private  Sector 
December  2004 

Business  Case  Analysis  and  Contractor  vs.  Organic  Support: 

A  First-Principles  View 

September  2004 

NPS-CM-04-006 

Measurement  Issues  in  Performance  Based  Logistics 

June  2004 

NPS-CM-04-004 

Update  of  the  Navy  Contract  Writing,  Phase  II 

June  2004 

NPS-CM-04-001 

Update  of  the  Navy  Contract  Writing,  Phase  1 

December  2003 

NPS-CM-04-002 

Marine  Corps  Contingency  Contracting  MCI 

December  2003 

Working  Paper  Series 

NPS-CM-04-016  An  Analysis  of  Industry’s  Perspective  on  the 


NPS-CM-04-012 

Recent  Changes  to  Circular  A-76 

October  2004 

Air  Force  Commodity  Councils: 

Leveraging  the  Power  of  Procurement 

September  2004 

NPS-CM-04-01 1 

Using  Metrics  to  Manage  Contractor  Performance 

September  2004 

NPS-LM-04-009 

Improving  the  Management  of  Reliability 

August  2004 

NPS-AM-04-007  The  Impact  of  Software  Support  on  System  Total  Ownership  Cost 
July  2004 

NPS-LM-04-003  Enablers  to  Ensure  a  Successful  Force  Centric  Logistics  Enterprise 
April  2004 

Acquisition  Case  Series 


NPS-CM-04-008 

Privatization  of  the  Naval  Air  Warfare  Center 

Aircraft  Division,  Indianapolis 

July  2004 

NPS-PM-04-010 

The  Army  Seeks  a  World  Class  Logistics  Modernization  Program 
June  2004 

Acquisition  Symposium  Proceedings 

NPS-AM-04-005  Charting  a  Course  for  Change:  Acquisition  Theory  and  Practice  for 
a  Transforming  Defense 
May  2004 

FY  2003  Sponsored  Acquisition  Research  Products 

Sponsored  Report  Series 


NPS-AM-03-003 

Centralized  Control  of  Defense  Acquisition  Programs: 

A  Comparative  Review  of  the  Framework  from  1987  -  2003 
September  2003 

NPS-AM-03-004 

Reduction  of  Total  Ownership  Cost 

September  2003 

NPS-CM-03-006 

Auto-Redact  Toolset  for  Department  of  Defense  Contracts 
September  2003 

Working  Paper  Series 

NPS-CM-03-002  Transformation  in  DOD  Contract  Closeout 
June  2003 

Acquisition  Case  Series 

NPS-CM-03-005  Contract  Closeout  (A) 

September  2003 

Other  Sponsored  Research 

NPS-CM-03-001  Transformation  in  DOD  Contract  Closeout 
MBA  Professional  Report 
June  2003 


Copies  of  the  Acquisition  Sponsored  Research  Reports  may  be  printed  from  our 
website  www.nps.navy.mil/qsbpp/acqn/publications 
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